Loading ...
Sorry, an error occurred while loading the content.

4910Re: [XP] Role of QA

Expand Messages
  • kent.schnaith@westgroup.com
    May 3 4:22 PM
      The best measure I have found for how development is progressing
      is "the number of open defects that need to be fixed before you can
      release the product". This measure is tracked from week to week,
      (or even day to day). The shape of the trend is usually a hump with
      a long tail. Early on, more defects will be discovered than
      are fixed, later few new defects should be discovered and the
      developers will catch up and reduce the backlog. Of course, if
      you have trouble fixing a problem without creating another problem
      then you are in for a very long march.

      Rewards should be based on delivering a quality product, on time.
      The statistically inclined can predict the expected duration of the

      One attraction of XP is that the focus on finding defects early and
      refactoring prevent you from falling into a long(endless) test and
      fix cycle.

      --- In extremeprogramming@egroups.com, Jen Wu <jen@d...> wrote:
      > The number of bugs found might be a good measure of how development
      > progressing, but I hate it as a metric for testers. It encourages
      > folks to write more bugs than are necessary. Instead, I'd suggest
      > that QA be measured on the scope of their tests and the accuracy of
      > the results. Number of test cases run, percentage of code covered,
      > percent of functionality covered, number of platforms tested, etc.
      > Bugs that are found by someone other than QA may be a measure of how
      > they could improve in the same way that bugs found by QA is a
      > of how development could improve. Also, bugs in QA code could count
      > against them (after all, QA is often a development project in and of
      > itself).
      > Jen
      > Duncan Gibson wrote:
      > >
      > > David Brady wrote:
      > > DB> My take on XP is that it's ABOUT quality. I don't stop work
      > > DB> on a unit until I believe that it can't be broken. If you
      > > DB> have no QA team, that's your only hope. If you have a good
      > > DB> QA team, it's easy to get lazy and write lots of hopey code,
      > > DB> because if it doesn't work, QA should catch it. In a better
      > > DB> world--one I believe *can* exist, but so far haven't seen,
      > > DB> and therefore am trying to create--the developers write the
      > > DB> Quality, and the (good) QA team does the Assurance.
      > >
      > > Many of the XP processes balance each other, or provide constant
      > > tension between them so that things run smoothly. It seems to me
      > > that XP offers the opportunity for the same type of balance, or
      > > constructive tension, between the developers and the QA/testing
      > > group.
      > >
      > > The current methodologies which offer BigBangIntegration and then
      > > provide [alpha and] beta versions of software, suffer from the
      > > problem that the overall team accepts and expects that the first
      > > release(s) will contain proportionally more defects than later
      > > ones, and that it will be part of the team's task to iron out
      > > these problems over time. The number of defects in the product
      > > should decrease with each release. Many developers don't test
      > > their own code adequately because they [wrongly] subscribe to the
      > > point of view that it is the task of the QA/testing group to
      > > catch errors in the code.
      > >
      > > Various authors[*] stress that this isn't the way to produce high
      > > quality software, and that the individual developer should strive
      > > to produce defect free software. Some people go as far as to
      > > consider any defect discovered by the QA/testing group - or worse
      > > still - the end user or customer, as a failure on the part of the
      > > developer.
      > >
      > > Some attempts to improve the situation by offering rewards for
      > > defects detected and removed proved counter-productive. Wily
      > > developers introduced known defects so that QA/testing would find
      > > them, and the developers could "solve" them in order to benefit
      > > from the reward.
      > >
      > > With XP, there is no BigBangIntegration. There is a series of
      > > smaller development cycles, even down to the internal 3-week
      > > iterations. If you consider that each of these cycles delivers
      > > approximately the same amount of new code, and that the defect
      > > rate is constant at N defects/KLOC (or however you measure it),
      > > then each cycle will also deliver the same number of new defects.
      > >
      > > In XP, the practice of UnitTest/TestFirst is intended to reduce
      > > the number of defects which slip through to the release (so N
      > > should be smaller). In XP, after the first cycle, the QA/testing
      > > team should be able to give an estimate for N, basically because
      > > N defects should have been found. After this first cycle, there
      > > is the possibility of introducing constructive tension between
      > > the developers and the QA/testing team. The developers should be
      > > aiming to deliver fewer than N defects per cycle, and the
      > > QA/testing group should be aiming to discover more than N defects
      > > per cycle. This would give measurable goals for both sides,
      > > possibly with some reward structure. N is adjusted after each
      > > cycle, and defects discovered by the end user/customer count
      > > against the QA/testing group.
      > >
      > > Any comments?
      > >
      > > Cheers
      > > Duncan
      > >
      > > [*] Writing Solid Code, Maguire, ISBN 1-55615-551-4
      > > The Pragmatic Programmer, Hunt & Thomas, ISBN 020161622X
      > > Introduction to the PSP, Humphrey, ISBN 0201548097
      > > The Practice of Programming, Kernigan & Pike, ISBN 020161586X
      > >
      > > This is my article, not my employer's, with my opinions and my
      > > --
      > > Duncan Gibson, ESTEC/TOS/MCV, Postbus 299, 2200AG Noordwijk,
      > > Tel: +31 71 5654013 Fax: +31 71 5656142 Email: duncan@y...
      > >
      > > To Post a message, send it to: extremeprogramming@e...
      > >
      > > To Unsubscribe, send a blank message to: extremeprogramming-
      > >
      > > Ad-free courtesy of objectmentor.com
    • Show all 20 messages in this topic