Loading ...
Sorry, an error occurred while loading the content.

RE: [XP] Role of QA

Expand Messages
  • Steve Goodhall
    My view would be that the best metric for testers is the ratio of total bugs to bugs found in production. This will give you a reasonable measurement of
    Message 1 of 20 , May 3, 2000
    • 0 Attachment
      My view would be that the best metric for testers is the ratio of total bugs
      to bugs found in production. This will give you a reasonable measurement of
      testing effectiveness and it is easy to maintain.

      Steve Goodhall
      Principal Architect
      QASolutions
      Compuware Corporation
      mailto:SGoodhall@...
      mailto:steve.goodhall@...
      http://members.home.net/sgoodhall/


      Victory awaits those who have everything in order. People call this luck. -
      Roald Amundsen





      > -----Original Message-----
      > From: Jen Wu [mailto:jen@...]
      > Sent: Wednesday, May 03, 2000 12:14 PM
      > To: extremeprogramming@egroups.com
      > Subject: Re: [XP] Role of QA
      >
      >
      > The number of bugs found might be a good measure of how development is
      > progressing, but I hate it as a metric for testers. It encourages QA
      > folks to write more bugs than are necessary. Instead, I'd suggest
      > that QA be measured on the scope of their tests and the accuracy of
      > the results. Number of test cases run, percentage of code covered,
      > percent of functionality covered, number of platforms tested, etc.
      >
      > Bugs that are found by someone other than QA may be a measure of how
      > they could improve in the same way that bugs found by QA is a measure
      > of how development could improve. Also, bugs in QA code could count
      > against them (after all, QA is often a development project in and of
      > itself).
      >
      > Jen
      >
      > Duncan Gibson wrote:
      > >
      > > David Brady wrote:
      > > DB> My take on XP is that it's ABOUT quality. I don't stop work
      > > DB> on a unit until I believe that it can't be broken. If you
      > > DB> have no QA team, that's your only hope. If you have a good
      > > DB> QA team, it's easy to get lazy and write lots of hopey code,
      > > DB> because if it doesn't work, QA should catch it. In a better
      > > DB> world--one I believe *can* exist, but so far haven't seen,
      > > DB> and therefore am trying to create--the developers write the
      > > DB> Quality, and the (good) QA team does the Assurance.
      > >
      > > Many of the XP processes balance each other, or provide constant
      > > tension between them so that things run smoothly. It seems to me
      > > that XP offers the opportunity for the same type of balance, or
      > > constructive tension, between the developers and the QA/testing
      > > group.
      > >
      > > The current methodologies which offer BigBangIntegration and then
      > > provide [alpha and] beta versions of software, suffer from the
      > > problem that the overall team accepts and expects that the first
      > > release(s) will contain proportionally more defects than later
      > > ones, and that it will be part of the team's task to iron out
      > > these problems over time. The number of defects in the product
      > > should decrease with each release. Many developers don't test
      > > their own code adequately because they [wrongly] subscribe to the
      > > point of view that it is the task of the QA/testing group to
      > > catch errors in the code.
      > >
      > > Various authors[*] stress that this isn't the way to produce high
      > > quality software, and that the individual developer should strive
      > > to produce defect free software. Some people go as far as to
      > > consider any defect discovered by the QA/testing group - or worse
      > > still - the end user or customer, as a failure on the part of the
      > > developer.
      > >
      > > Some attempts to improve the situation by offering rewards for
      > > defects detected and removed proved counter-productive. Wily
      > > developers introduced known defects so that QA/testing would find
      > > them, and the developers could "solve" them in order to benefit
      > > from the reward.
      > >
      > > With XP, there is no BigBangIntegration. There is a series of
      > > smaller development cycles, even down to the internal 3-week
      > > iterations. If you consider that each of these cycles delivers
      > > approximately the same amount of new code, and that the defect
      > > rate is constant at N defects/KLOC (or however you measure it),
      > > then each cycle will also deliver the same number of new defects.
      > >
      > > In XP, the practice of UnitTest/TestFirst is intended to reduce
      > > the number of defects which slip through to the release (so N
      > > should be smaller). In XP, after the first cycle, the QA/testing
      > > team should be able to give an estimate for N, basically because
      > > N defects should have been found. After this first cycle, there
      > > is the possibility of introducing constructive tension between
      > > the developers and the QA/testing team. The developers should be
      > > aiming to deliver fewer than N defects per cycle, and the
      > > QA/testing group should be aiming to discover more than N defects
      > > per cycle. This would give measurable goals for both sides,
      > > possibly with some reward structure. N is adjusted after each
      > > cycle, and defects discovered by the end user/customer count
      > > against the QA/testing group.
      > >
      > > Any comments?
      > >
      > > Cheers
      > > Duncan
      > >
      > > [*] Writing Solid Code, Maguire, ISBN 1-55615-551-4
      > > The Pragmatic Programmer, Hunt & Thomas, ISBN 020161622X
      > > Introduction to the PSP, Humphrey, ISBN 0201548097
      > > The Practice of Programming, Kernigan & Pike, ISBN 020161586X
      > >
      > > This is my article, not my employer's, with my opinions and my
      > disclaimer!
      > > --
      > > Duncan Gibson, ESTEC/TOS/MCV, Postbus 299, 2200AG Noordwijk, Netherlands
      > > Tel: +31 71 5654013 Fax: +31 71 5656142 Email: duncan@...
      > >
      > > To Post a message, send it to: extremeprogramming@...
      > >
      > > To Unsubscribe, send a blank message to:
      > extremeprogramming-unsubscribe@...
      > >
      > > Ad-free courtesy of objectmentor.com
      >
      >
      > To Post a message, send it to: extremeprogramming@...
      >
      > To Unsubscribe, send a blank message to:
      > extremeprogramming-unsubscribe@...
      >
      > Ad-free courtesy of objectmentor.com
      >
    Your message has been successfully submitted and would be delivered to recipients shortly.