Loading ...
Sorry, an error occurred while loading the content.

[extremeprogramming] extreme STD's (Stupidly Transmitted Defects)

Expand Messages
  • Chris Cunningham
    Scenario: A defect is found in release A and corrected in Patch A.1 The same defect is reintroduced in release B because Patch A.1 was not merged into the
    Message 1 of 4 , Feb 1, 2000
    • 0 Attachment
      Scenario:
      A defect is found in release A and corrected in Patch A.1
      The same defect is reintroduced in release B because Patch A.1
      was not merged into the build. The test case was run but the
      results were a false positive (bad test case, no way to tie the
      test case to the original defect so the results were not reviewed
      correctly).

      People:
      1) Programmer + pair used wrong view in their personnel cm space
      2) team lead missed it during the master merge
      3) Cm lead missed it again during audit
      4) Unit and Integration ran tests but misunderstood results
      5) QA ran tests but assumed it passed
      6) User Acceptance changed test so results would not have caught the error
      All were asked to report status and reported that the tests ran ok.

      Question:
      1) What else was missing in the process?
      2) How to deal with the individuals involved?
      3) What would you have done differently?

      Chris Cunningham
      chrisc@...
    • Andy Glew
      Message 2 of 4 , Feb 1, 2000
      • 0 Attachment
        Chris Cunningham wrote:
        >
        > Scenario:
        > A defect is found in release A and corrected in Patch A.1
        > The same defect is reintroduced in release B because Patch A.1
        > was not merged into the build. The test case was run but the
        > results were a false positive (bad test case, no way to tie the
        > test case to the original defect so the results were not reviewed
        > correctly).
      • Andy Glew
        Blarg.... sorry about that. My keyboard got screwed up, starting spewing garbage, and needed to be rebooted. ... If the test produced a false positive when run
        Message 3 of 4 , Feb 1, 2000
        • 0 Attachment
          Blarg.... sorry about that. My keyboard got screwed up, starting
          spewing garbage, and needed to be rebooted.

          Had meant not to say anything, but since I have:


          > Chris Cunningham wrote:
          > >
          > > Scenario:
          > > A defect is found in release A and corrected in Patch A.1
          > > The same defect is reintroduced in release B because Patch A.1
          > > was not merged into the build. The test case was run but the
          > > results were a false positive.

          If the test produced a false positive when run against release A,
          *that* was your problem: not running the test against release A.
          Too many programmers write tests after the fix has been made and
          merged, and, as a result, never verify that the test actually discovers
          the bug. One of the nice things about XP is that it emphasizes writing
          tests before you write the code.

          If the test failed correctly against release A, passed on patch release A.1,
          but gave an incorrect false positive against release B, then,
          sigh, you're in a sad place.

          What it probably means is that the test tested for a different bug.
          This bug probably got fixed as a side effect of A.1 and B, but the
          original bug was still present in B. Of course, it's also possible that the
          test is just plain broken, randomly passing and failing.

          As in most programming problems, code review is the best safeguard.
          Tests need to be reviewed like any other piece of code. Pair programming
          is continuous review. Even test cases need to be pair programmed... ?

          (I'd also suggest using a test coverage tool to verify that the test covered
          the lines of code / conditions / whatever needed to be changed between
          A and A.1. But that's an orthogonal issue to XP.)
        • Ron Jeffries
          ... I d like to see the code that can do this thing. Ron Jeffries Extreme Programming Training and Consultation www.XProgramming.com
          Message 4 of 4 , Feb 1, 2000
          • 0 Attachment
            At 05:02 PM 2/1/2000 -0600, you wrote:
            >If the test failed correctly against release A, passed on patch release A.1,
            >but gave an incorrect false positive against release B, then,
            >sigh, you're in a sad place.

            I'd like to see the code that can do this thing.

            Ron Jeffries
            Extreme Programming Training and Consultation
            www.XProgramming.com
          Your message has been successfully submitted and would be delivered to recipients shortly.