Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Re: Role of QA

Expand Messages
  • Peter D. Padilla
    Is QA the customer? Is QA part of development? First, I d really hesitate to have the customer perform functional or acceptance tests before QA (or the role
    Message 1 of 1 , May 2, 2000
      Is QA the customer? Is QA part of development?

      First, I'd really hesitate to have the customer perform functional or
      acceptance tests before QA (or the role of QA) has had a chance to aid in
      getting some "kinks" worked out of the system. Code that "works" at a unit
      and even basic functional level often (usually?) still has enough bugs or
      misconceived processes that it will only undermine the customer's faith in
      the development team if the customer sees it immediately.

      I like the idea that QA advises, participates in test
      planning/documentation/metrics/defect management (our VP of development
      calls them "variances"), and keeps appropriate metrics to prove the process
      success. Our product doesn't have much of a user interface - it's a
      back-end system integration product - so, it really is up to QA and a group
      of installers to act as the "customer" because the actual customer only sees
      the input/output aspects of the system.

      In the past I have advocated QA being very separate from development so that
      QA/test is not a "rubber stamp" organization, simply "approving" everything
      development throws over the wall. Currently, I'd like to see the two groups
      more intimately connected - and I'm considering putting a QA Analyst on each
      development team, to help write functional and acceptance tests, coordinate
      and track testing by installers (customers?), ensure that we are not losing
      quality but are gaining it by employing RAD techniques - XP - and so forth.
      Is it extraneous? Time will tell, I suppose... but the QA Analyst may also
      carry some "project administrator" responsibilities, since we will not have
      a project manager "overseeing" the work... we have enough departmental
      managers to handle the administrivia, and we work closely with product

      We'll see...
      ----- Original Message -----
      From: Lowell Lindstrom <lindstrom@...>
      To: <extremeprogramming@egroups.com>
      Sent: Tuesday, May 02, 2000 8:58 AM
      Subject: [XP] Re: Role of QA

      > I am responding many of the "Role of QA" messages from the Digest this
      > morning:
      > > Date: Mon, 01 May 2000 19:36:43 -0700
      > > From: Jen Wu <jen@...>
      > >Subject: Role of QA
      > >
      > >I don't know if a lot has been said for the role of QA, but here are
      > >some questions ...
      > >
      > Acceptance of the system is the responsibility of the customer. Only
      > the customer can define the cost/quality trade-off. Programmers cannot
      > deliver crap, of course, but given the difference in required quality
      > levels between a life-critical system and release 1.0 of my latest
      > e-commerce experiment, decisions must be made as to how much quality
      > should be designed and tested in. The customer must drive this decision
      > through the conversation with the programmers.
      > XP expands the responsibilities of the customer in this regard. My
      > personal past experience has been that customer acceptance is a cursory
      > check of major functionality. With XP, they write all of the functional
      > tests. This likely means providing resources to the customer that they
      > are not used to having.
      > >Some background ... a sophisticated QA team will do most if not all of
      > >the following (among other things):
      > >
      > >* Develop a test plan, including test suites and cases
      > >* Structured black box testing -- tested by hand
      > >* Ad hoc black box testing
      > >* Structured automated functional testing -- testing using automated
      > > tools on the UI (no calls to code)
      > >* White box and intrusive automated tests (code reviews and tests
      > > like the unit tests that the programmers are responsible for in
      > > XP)
      > >* Code coverage
      > >* Bug tracking (correlated with test cases and code coverage)
      > >* Multi-user and performance testing using testing tools
      > >
      > Most of this should be done by the customer or under the customer's
      > direction. The amount of any of this that you do should be driven by
      > the customer's quality requirements, which will likely be driven by
      > their balance between cost/time pressures to release versus cost of
      > defects after release.
      > I am not sure what best practice is concerning bug tracking in terms
      > customer or programmer ownership. I have heard of bugs turning into
      > user stories and being dealt with by the planning game in subsequent
      > iterations/releases.
      > A note on code coverage. Test-first programming will ensure high code
      > coverage at the unit test level. The biggest benefit I can see from code
      > coverage analysis at the functional test level is helping to identify
      > code that can be refactored out of the system.
      > >With XP, there is some overlap between what a QA department
      > >would do and
      > >what developers would do. Should the QA people be brought into the
      > >development team?
      > >If so, what about the idea that a product should be thoroughly
      > >tested by
      > >someone who didn't take part in writing it?
      > >If not ...
      > >Should QA and development work on the same testing framework?
      > I believe they should moved towards the customer, rather than the
      > developers.
      > There are some interesting functional test frameworks emerging from XP
      > projects. I have also heard of goals to extend JUnit for functional
      > testing. Others that are more closely involved will need to add details
      > here.
      > >When should QA start being involved?
      > Customers begin writing functional tests as soon as the user stories for
      > the iteration have been defined. QA (as part of the customer) must be
      > involved at that time.
      > >Message: 7
      > > Date: Tue, 2 May 2000 00:13:41 -0500
      > > From: "Peter D. Padilla" <padillap@...>
      > >Subject: Re: Role of QA
      > >
      > >
      > >Look, I've only been lurking for a short while... and my total
      > >experience
      > >with XP is from reading the book three weeks ago. But if it
      > >boils down to
      > >developers wanting to throw a lot of valuable process out the
      > >window because
      > >it "restricts" them, maybe I'm looking at the wrong
      > >alternative to enable
      > >great applications. I don't want to be offensive, and I apologize if
      > >anything in this message is perceived as derogatory... but a great team
      > >knows how to delegate its responsibilities - even those which aren't
      > >strictly development.
      > I think Michael's views reflect an experience that many have had of an
      > external QA group not being driven by the customer's requirements.
      > As with many XP practices, the goal is not throw good practices out the
      > window, but rather to focus them on delivering value to customers
      > through the work of empowered programmers. The system must be accepted,
      > but the customer needs to drive what acceptance means. In many cases,
      > this will require lot's of QA, but not because of an external QA
      > engineer/manager's or process document's definition of quality, but
      > because of the customer's definition of quality.
      > >Message: 8
      > > Date: Tue, 02 May 2000 11:22:03 +0200
      > > From: Malte Kroeger <kroeger@...>
      > >Subject: Re: Role of QA
      > >
      > >What XP sais (as I understand it):
      > >
      > >XP doesn't officially have a QA-Team. In XP the developers do all the
      > >tests. They write their own unit tests, of course, and they support the
      > >customer in working out the functional tests.
      > >
      > >The question is: Is this a good way to do it? Is this enough testing?
      > As described,IMHO, this is not XP. In XP, the developers do all the
      > UNIT tests. Acceptance of the system is the responsibility of the
      > customer. In this regard, XP places more responsibility on the customer
      > to say what is acceptable. XP does not describe a QA-team, but the
      > function and responsibilities don't disappear, they become part of the
      > customer's role.
      > >
      > >In many development teams now there is a lot less testing than this, so
      > >doing it the XP way would be an advantage already.
      > >
      > Agreed, we're seeing this in many of the projects that we coach today.
      > The biggest short term benefit is the improved testing.
      > >But what I think is most important about a QA-Team is, that it stays in
      > >very close contact with the development team.
      > Yes, the customer, as well.
      > >What is really needed is, that QA-Team members pair with developers for
      > >writing tests. That they review the tests done so far. That they make
      > >suggestions on how to improve the tests and do the improvements pairing
      > >with developers.
      > >
      > >In big systems it might be helpful to have some people of the
      > >QA-Team do
      > >some adhoc-manual testing of the integrated system as well,
      > >since not all
      > >the tests can be fully automated in a reasonable amount of time.
      > >
      > >So from my point of view the QA-Team should do less testing themselves,
      > >but should give more support to the developers in doing the
      > >tests and in
      > >developing a high quality product in the first place.
      > >
      > >So it's more of a QA-Coaching role.
      > I am not sure I agree with the pairing of QA/developers. XP is trying
      > to maximize the focus of the developers on delivering valuable
      > functionality. But it is interesting proposal. I'd be interested in
      > others experience here.
      > Lowell
      > To Post a message, send it to: extremeprogramming@...
      > To Unsubscribe, send a blank message to:
      > Ad-free courtesy of objectmentor.com
    Your message has been successfully submitted and would be delivered to recipients shortly.