Loading ...
Sorry, an error occurred while loading the content.

4852Re: Role of QA

Expand Messages
  • Lowell Lindstrom
    May 2, 2000
      I am responding many of the "Role of QA" messages from the Digest this

      > Date: Mon, 01 May 2000 19:36:43 -0700
      > From: Jen Wu <jen@...>
      >Subject: Role of QA
      >I don't know if a lot has been said for the role of QA, but here are
      >some questions ...

      Acceptance of the system is the responsibility of the customer. Only
      the customer can define the cost/quality trade-off. Programmers cannot
      deliver crap, of course, but given the difference in required quality
      levels between a life-critical system and release 1.0 of my latest
      e-commerce experiment, decisions must be made as to how much quality
      should be designed and tested in. The customer must drive this decision
      through the conversation with the programmers.

      XP expands the responsibilities of the customer in this regard. My
      personal past experience has been that customer acceptance is a cursory
      check of major functionality. With XP, they write all of the functional
      tests. This likely means providing resources to the customer that they
      are not used to having.

      >Some background ... a sophisticated QA team will do most if not all of
      >the following (among other things):
      >* Develop a test plan, including test suites and cases
      >* Structured black box testing -- tested by hand
      >* Ad hoc black box testing
      >* Structured automated functional testing -- testing using automated
      > tools on the UI (no calls to code)
      >* White box and intrusive automated tests (code reviews and tests
      > like the unit tests that the programmers are responsible for in
      > XP)
      >* Code coverage
      >* Bug tracking (correlated with test cases and code coverage)
      >* Multi-user and performance testing using testing tools

      Most of this should be done by the customer or under the customer's
      direction. The amount of any of this that you do should be driven by
      the customer's quality requirements, which will likely be driven by
      their balance between cost/time pressures to release versus cost of
      defects after release.

      I am not sure what best practice is concerning bug tracking in terms
      customer or programmer ownership. I have heard of bugs turning into
      user stories and being dealt with by the planning game in subsequent

      A note on code coverage. Test-first programming will ensure high code
      coverage at the unit test level. The biggest benefit I can see from code
      coverage analysis at the functional test level is helping to identify
      code that can be refactored out of the system.

      >With XP, there is some overlap between what a QA department
      >would do and
      >what developers would do. Should the QA people be brought into the
      >development team?
      >If so, what about the idea that a product should be thoroughly
      >tested by
      >someone who didn't take part in writing it?
      >If not ...
      >Should QA and development work on the same testing framework?

      I believe they should moved towards the customer, rather than the

      There are some interesting functional test frameworks emerging from XP
      projects. I have also heard of goals to extend JUnit for functional
      testing. Others that are more closely involved will need to add details

      >When should QA start being involved?

      Customers begin writing functional tests as soon as the user stories for
      the iteration have been defined. QA (as part of the customer) must be
      involved at that time.

      >Message: 7
      > Date: Tue, 2 May 2000 00:13:41 -0500
      > From: "Peter D. Padilla" <padillap@...>
      >Subject: Re: Role of QA
      >Look, I've only been lurking for a short while... and my total
      >with XP is from reading the book three weeks ago. But if it
      >boils down to
      >developers wanting to throw a lot of valuable process out the
      >window because
      >it "restricts" them, maybe I'm looking at the wrong
      >alternative to enable
      >great applications. I don't want to be offensive, and I apologize if
      >anything in this message is perceived as derogatory... but a great team
      >knows how to delegate its responsibilities - even those which aren't
      >strictly development.

      I think Michael's views reflect an experience that many have had of an
      external QA group not being driven by the customer's requirements.

      As with many XP practices, the goal is not throw good practices out the
      window, but rather to focus them on delivering value to customers
      through the work of empowered programmers. The system must be accepted,
      but the customer needs to drive what acceptance means. In many cases,
      this will require lot's of QA, but not because of an external QA
      engineer/manager's or process document's definition of quality, but
      because of the customer's definition of quality.

      >Message: 8
      > Date: Tue, 02 May 2000 11:22:03 +0200
      > From: Malte Kroeger <kroeger@...>
      >Subject: Re: Role of QA
      >What XP sais (as I understand it):
      >XP doesn't officially have a QA-Team. In XP the developers do all the
      >tests. They write their own unit tests, of course, and they support the
      >customer in working out the functional tests.
      >The question is: Is this a good way to do it? Is this enough testing?

      As described,IMHO, this is not XP. In XP, the developers do all the
      UNIT tests. Acceptance of the system is the responsibility of the
      customer. In this regard, XP places more responsibility on the customer
      to say what is acceptable. XP does not describe a QA-team, but the
      function and responsibilities don't disappear, they become part of the
      customer's role.

      >In many development teams now there is a lot less testing than this, so
      >doing it the XP way would be an advantage already.

      Agreed, we're seeing this in many of the projects that we coach today.
      The biggest short term benefit is the improved testing.

      >But what I think is most important about a QA-Team is, that it stays in
      >very close contact with the development team.

      Yes, the customer, as well.

      >What is really needed is, that QA-Team members pair with developers for
      >writing tests. That they review the tests done so far. That they make
      >suggestions on how to improve the tests and do the improvements pairing
      >with developers.
      >In big systems it might be helpful to have some people of the
      >QA-Team do
      >some adhoc-manual testing of the integrated system as well,
      >since not all
      >the tests can be fully automated in a reasonable amount of time.
      >So from my point of view the QA-Team should do less testing themselves,
      >but should give more support to the developers in doing the
      >tests and in
      >developing a high quality product in the first place.
      >So it's more of a QA-Coaching role.

      I am not sure I agree with the pairing of QA/developers. XP is trying
      to maximize the focus of the developers on delivering valuable
      functionality. But it is interesting proposal. I'd be interested in
      others experience here.

    • Show all 20 messages in this topic