Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Functional Testing

Expand Messages
  • Ron Jeffries
    (thanks for pointing out where the questions were, I guess I stopped reading in awe when I got to my name ...) ... Your intuition about what should be done is
    Message 1 of 4 , Apr 3, 2000
    • 0 Attachment
      (thanks for pointing out where the questions were, I guess I stopped
      reading in awe when I got to my name ...)

      At 03:43 PM 3/29/2000 -0700, Dean.Billigmeier@... wrote:
      >The perception I see in the XP camp is that the functional tests will be
      >developed, debugged and running prior to the first piece of code hitting
      >the test platform. How so?

      Your intuition about what should be done is right on, shown in what you
      write here and below. I see that I've not been clear in what I recommend,
      or in my thinking.

      But still, having all the tests isn't the perception I have or teach. We do
      expect the tests to grow in number over time. The picture in the article at
      http://www.xprogramming.com/publications/software_testing.htm shows the
      test count and percentage correct growing over time. That's what we usually
      see.

      It would be great if we got FTs as we went along that should score at 100%
      by the end of the iteration they apply to. That seems rarely to happen, as
      the customers prefer to test the full business purpose of the app, even on
      simple data, from the beginning. Your thoughts, however, lead me to want to
      be more intelligent in working on setting up the tests.

      If they _can_ provide tests that will score 100%, that would be great. We
      want the tests to be permanent, however, so IMO it is better to get a test
      that, though simple, will last for "all time", rather than having to be
      thrown away and replaced. I would certainly love to have tests intended to
      score at 100% right along.

      I wouldn't want to put tests into the mix that couldn't POSSIBLY work.
      Sometimes you just have to. You're right not to want to, and your
      contribution here makes me want to sign up for your ideas rather than my own.

      >I can envision keeping pace or just ahead of
      >the code development, but not finished. The first runs of the functional
      >tests will require careful analysis to determine where the fault is: the
      >test code or the application code. I have committed to engineering that in
      >the early stages we will presume the test is at fault until proven other
      >wise. If after the post-run debrief & results analysis we think the test
      >is correct, we'll then consult with engineering.

      Sounds right to me. If you are evolving the test framework, you should be
      sure that IT isn't causing problems early on. Then there is the possibility
      of tests with bad answers, which hopefully will be few. So its most
      probable that the system is wrong, and anyway, as you clearly see, it's the
      best position to take politically.

      > I understand why an end to end functional test would not score 100% in the
      >beginning. What escapes me is WHY would a test manager:
      > Waste test time running a battery of functional tests of which only 5%
      > (chosen @random) are expected to pass?

      It would be better if more than 5% were expected to pass. Ideal if all, of
      course.

      > Publish the result : 5% of the functional tests passed. Sounds real
      > encouraging to engineering.

      It'll be encouraging tomorrow when it goes up to 6 ...

      >In reality, if 5% of the total
      > functionality was placed into the build, and all functional tests for
      > the delivered functionality passed, then success is 100%. THATs
      > something to party about.

      Yes, that would be ideal. We are in agreement. If you can get this, take it.

      >Now help me understand "Each test exercises the whole system, end to end".
      >Why should engineering code be modular and not functional tests? Create
      >modular test which exercise a specific function, for instance, in the
      >telephony industry, the ability for a caller to leave a message. As more
      >functionality is built, more of the functional test suite is "turned on".

      If you can do this, go for it. My point, however, was slightly different.
      Your functional tests need to exercise the system from its rawest (then)
      inputs to its most final (then) outputs, rather than exercise internal
      components. The sooner you get out to the edges, the sooner you'll find things.

      As for creating more modular tests, I think you're beyond us on the
      smartness scale, in a good way. I've not thought as much as I should about
      finding ways to make the tests more likely to run at 100, and yet preserve
      their correctness over time. You're on the right track, and if it doesn't
      slow you down - and I don't think it will - go for it.

      >This approach strikes me as being:
      > Supportive of XP development style
      > Modular, keeping pace with engineering
      > Efficient
      > Encouraging to Engineering

      Absolutely. Now that I see your comments (sorry), I'm very much in accord
      with what you're trying to do. As long as it just helps, go for it!

      Thanks and regards,

      Ron Jeffries
      www.XProgramming.com
    Your message has been successfully submitted and would be delivered to recipients shortly.