Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] How do you handle defect resolution?

Expand Messages
  • Keith Ray
    correctIon: ... these tests would have to PASS for the task to be acceptable at the end of the sprint. ... -- C. Keith Ray, IXP Coach, Industrial Logic,
    Message 1 of 18 , Jan 2, 2009
    • 0 Attachment
      correctIon: "... these tests would have to PASS for the task to be
      acceptable at the end of the sprint."

      On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray <keith.ray@...> wrote:
      > Well, I'm pretty old. I would recommend creating automated tests that
      > fail because of the defect, and these tests would have to fail for the
      > task to be acceptable at the end of the sprint.
      >
      > Think about how the functionality accumulates with each sprint - there
      > will be soon no time for manual re-testing of previously implemented
      > functionality. Without test automation, there could be undetected
      > regressions.
      >
      > Some dialogs with the tester, development team and product owners may
      > result in a "defect" being classified as a new backlog item, to be
      > implemented a later sprint. This is particularly relevant for
      > "missing" functionality.
      >
      >
      > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...> wrote:
      >> In our sprints I have a QA person manually testing bits of functionality as
      >> they are delivered; This generates defects as we go and we've been adding a
      >> task to the Sprint backlog for each defect. I'm not completely satisfied
      >> with this approach, so I am asking what other, more experienced folks are
      >> doing with defects.
      >> Thanks,
      >>
      >> --
      >> http://mikeonitstuff.net/ New Blog
      >> http://mikeonitstuff.com/ Old Blog
      >> http://mikeonbikes.blogspot.com/
      >>
      >>
      >> [Non-text portions of this message have been removed]
      >>
      >>
      >> ------------------------------------
      >>
      >> To Post a message, send it to: extremeprogramming@...
      >>
      >> To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
      >>
      >> ad-free courtesy of objectmentor.comYahoo! Groups Links
      >>
      >>
      >>
      >>
      >
      >
      >
      > --
      > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
      > http://industriallogic.com 866-540-8336 (toll free)
      > Groove with our Agile Greatest Hits: http://www.industriallogic.com/elearning/
      > http://agilesolutionspace.blogspot.com/
      >



      --
      C. Keith Ray, IXP Coach, Industrial Logic, Inc.
      http://industriallogic.com 866-540-8336 (toll free)
      Groove with our Agile Greatest Hits: http://www.industriallogic.com/elearning/
      http://agilesolutionspace.blogspot.com/
    • Mike Coon
      Thanks for the response - sometimes just talking about things brings clarity. Yes I agree with automated testing. For a variety of poor excuses we don t have
      Message 2 of 18 , Jan 2, 2009
      • 0 Attachment
        Thanks for the response - sometimes just talking about things brings
        clarity.

        Yes I agree with automated testing. For a variety of poor excuses we don't
        have much yet. I am talking about the churn of defect resolution where it
        seems we have a lot of stop/start as defects are opened, evaluated, fixed,
        and retested.
        I may use our unpleasant defect experience to make a better case to the dev
        managers for unit and functional testing. At this point they agree that it
        would be good but argue they don't have time. Predictably, we spend more
        time on defects than we would spend writing tests as we go.

        Most of our developers are not familiar with TDD at this point.

        Mike

        On Fri, Jan 2, 2009 at 12:09 PM, Keith Ray <keith.ray@...> wrote:

        > correctIon: "... these tests would have to PASS for the task to be
        >
        > acceptable at the end of the sprint."
        >
        > On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray <keith.ray@...<keith.ray%40gmail.com>>
        > wrote:
        > > Well, I'm pretty old. I would recommend creating automated tests that
        > > fail because of the defect, and these tests would have to fail for the
        > > task to be acceptable at the end of the sprint.
        > >
        > > Think about how the functionality accumulates with each sprint - there
        > > will be soon no time for manual re-testing of previously implemented
        > > functionality. Without test automation, there could be undetected
        > > regressions.
        > >
        > > Some dialogs with the tester, development team and product owners may
        > > result in a "defect" being classified as a new backlog item, to be
        > > implemented a later sprint. This is particularly relevant for
        > > "missing" functionality.
        > >
        > >
        > > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...<mcoon1961%40gmail.com>>
        > wrote:
        > >> In our sprints I have a QA person manually testing bits of functionality
        > as
        > >> they are delivered; This generates defects as we go and we've been
        > adding a
        > >> task to the Sprint backlog for each defect. I'm not completely satisfied
        > >> with this approach, so I am asking what other, more experienced folks
        > are
        > >> doing with defects.
        > >> Thanks,
        > >>
        > >> --
        > >> http://mikeonitstuff.net/ New Blog
        > >> http://mikeonitstuff.com/ Old Blog
        > >> http://mikeonbikes.blogspot.com/
        > >>
        > >>
        > >> [Non-text portions of this message have been removed]
        > >>
        > >>
        > >> ------------------------------------
        > >>
        > >> To Post a message, send it to: extremeprogramming@...<extremeprogramming%40eGroups.com>
        > >>
        > >> To Unsubscribe, send a blank message to:
        > extremeprogramming-unsubscribe@...<extremeprogramming-unsubscribe%40eGroups.com>
        > >>
        > >> ad-free courtesy of objectmentor.comYahoo! Groups Links
        > >>
        > >>
        > >>
        > >>
        > >
        > >
        > >
        > > --
        > > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
        > > http://industriallogic.com 866-540-8336 (toll free)
        > > Groove with our Agile Greatest Hits:
        > http://www.industriallogic.com/elearning/
        > > http://agilesolutionspace.blogspot.com/
        > >
        >
        > --
        > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
        > http://industriallogic.com 866-540-8336 (toll free)
        > Groove with our Agile Greatest Hits:
        > http://www.industriallogic.com/elearning/
        > http://agilesolutionspace.blogspot.com/
        >
        >
        >



        --
        http://mikeonitstuff.net/ New Blog
        http://mikeonitstuff.com/ Old Blog
        http://mikeonbikes.blogspot.com/


        [Non-text portions of this message have been removed]
      • Curt Coulter
        This is what I do. QA testing is happening in real time with stories being closed. Defects that materially affect the functionality being delivered, if
        Message 3 of 18 , Jan 2, 2009
        • 0 Attachment
          This is what I do. QA testing is happening in real time with stories
          being closed. Defects that materially affect the functionality being
          delivered, if missed by the automated tests, cause the story to 'test
          fail' and we of course do not get the points for that story until it
          is completely test passed. Any defects are closed as part of the
          story, using the priority order as set in the sprint tasks. In this
          case, the developers are to write a (failing) automated unit test
          first, and then fix the bug.

          If the defect is something ancillary, the product owner(s) and I
          evaluate it as to whether it is required for the story in question, or
          can go on the backlog for the next sprint.

          Ahh, the old 'we dont' have time' argument, it cracks me up at this
          point. They don't have time because they or the organization is
          measuring the wrong metrics of productivity and/or lean/agile is not
          in practice at any higher level than the development teams (if there).
          In my experience, nothing will change until quality becomes the
          primary focus.

          /Curt

          On Fri, Jan 2, 2009 at 12:22 PM, Mike Coon <mcoon1961@...> wrote:
          > Thanks for the response - sometimes just talking about things brings
          > clarity.
          >
          > Yes I agree with automated testing. For a variety of poor excuses we don't
          > have much yet. I am talking about the churn of defect resolution where it
          > seems we have a lot of stop/start as defects are opened, evaluated, fixed,
          > and retested.
          > I may use our unpleasant defect experience to make a better case to the dev
          > managers for unit and functional testing. At this point they agree that it
          > would be good but argue they don't have time. Predictably, we spend more
          > time on defects than we would spend writing tests as we go.
          >
          > Most of our developers are not familiar with TDD at this point.
          >
          > Mike
          >
          > On Fri, Jan 2, 2009 at 12:09 PM, Keith Ray <keith.ray@...> wrote:
          >
          >> correctIon: "... these tests would have to PASS for the task to be
          >>
          >> acceptable at the end of the sprint."
          >>
          >> On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray
          >> <keith.ray@...<keith.ray%40gmail.com>>
          >> wrote:
          >> > Well, I'm pretty old. I would recommend creating automated tests that
          >> > fail because of the defect, and these tests would have to fail for the
          >> > task to be acceptable at the end of the sprint.
          >> >
          >> > Think about how the functionality accumulates with each sprint - there
          >> > will be soon no time for manual re-testing of previously implemented
          >> > functionality. Without test automation, there could be undetected
          >> > regressions.
          >> >
          >> > Some dialogs with the tester, development team and product owners may
          >> > result in a "defect" being classified as a new backlog item, to be
          >> > implemented a later sprint. This is particularly relevant for
          >> > "missing" functionality.
          >> >
          >> >
          >> > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
          >> > <mcoon1961@...<mcoon1961%40gmail.com>>
          >> wrote:
          >> >> In our sprints I have a QA person manually testing bits of
          >> >> functionality
          >> as
          >> >> they are delivered; This generates defects as we go and we've been
          >> adding a
          >> >> task to the Sprint backlog for each defect. I'm not completely
          >> >> satisfied
          >> >> with this approach, so I am asking what other, more experienced folks
          >> are
          >> >> doing with defects.
          >> >> Thanks,
          >> >>
          >> >> --
          >> >> http://mikeonitstuff.net/ New Blog
          >> >> http://mikeonitstuff.com/ Old Blog
          >> >> http://mikeonbikes.blogspot.com/
          >> >>
          >> >>
          >> >> [Non-text portions of this message have been removed]
          >> >>
          >> >>
          >> >> ------------------------------------
          >> >>
          >> >> To Post a message, send it to:
          >> >> extremeprogramming@...<extremeprogramming%40eGroups.com>
          >> >>
          >> >> To Unsubscribe, send a blank message to:
          >>
          >> extremeprogramming-unsubscribe@...<extremeprogramming-unsubscribe%40eGroups.com>
          >> >>
          >> >> ad-free courtesy of objectmentor.comYahoo! Groups Links
          >> >>
          >> >>
          >> >>
          >> >>
          >> >
          >> >
          >> >
          >> > --
          >> > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
          >> > http://industriallogic.com 866-540-8336 (toll free)
          >> > Groove with our Agile Greatest Hits:
          >> http://www.industriallogic.com/elearning/
          >> > http://agilesolutionspace.blogspot.com/
          >> >
          >>
          >> --
          >> C. Keith Ray, IXP Coach, Industrial Logic, Inc.
          >> http://industriallogic.com 866-540-8336 (toll free)
          >> Groove with our Agile Greatest Hits:
          >> http://www.industriallogic.com/elearning/
          >> http://agilesolutionspace.blogspot.com/
          >>
          >>
          >>
          >
          > --
          > http://mikeonitstuff.net/ New Blog
          > http://mikeonitstuff.com/ Old Blog
          > http://mikeonbikes.blogspot.com/
          >
          > [Non-text portions of this message have been removed]
          >
          >
        • Adam Sroka
          ... I heard a description of how Google does this that I really liked. Basically, they have a concept of Certify in a Day . The idea is that the product needs
          Message 4 of 18 , Jan 2, 2009
          • 0 Attachment
            On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...> wrote:
            > In our sprints I have a QA person manually testing bits of functionality as
            > they are delivered; This generates defects as we go and we've been adding a
            > task to the Sprint backlog for each defect. I'm not completely satisfied
            > with this approach, so I am asking what other, more experienced folks are
            > doing with defects.

            I heard a description of how Google does this that I really liked.
            Basically, they have a concept of "Certify in a Day". The idea is that
            the product needs to be releasable at any time. So, if you find a bug
            you need to fix the bug the same day or you won't be able to "Certify
            in a Day." Of course, that is the goal and by their own admission they
            don't always hit it and some product teams aren't anywhere close.
            Still, it is a good way of keeping the fix close to the feature. Also,
            Google depends heavily on automated tests and continuous builds to
            enable this kind of rapidity.
          • Mike Coon
            Yeah, I ve been having a hard time getting traction on both test and build/deploy automation; which seems odd given that we re all in the automation business.
            Message 5 of 18 , Jan 2, 2009
            • 0 Attachment
              Yeah, I've been having a hard time getting traction on both test and
              build/deploy automation; which seems odd given that we're all in the
              automation business. Dev managers start with the - I think incorrect -
              assumption that business people can't/don't/won't see the value of these
              "infrastructure" activities.
              The automation is the key for our situation because if I can take the time
              and ambiguity of out deployment and testing, some other thing can become the
              new bottleneck.

              One thing that happens now is we get defects from UAT folks that have not
              gotten used to the idea that their defect should include steps, expected,
              and actual results. So we spend a lot of time figuring out what they are
              reporting which often turns out to be a feature request, training issue, or
              user configuration setting - all of which do need to be identified and
              resolved but currently take too long to understand, categorize, and address.

              Thanks, Mike

              On Fri, Jan 2, 2009 at 1:33 PM, Adam Sroka <adam.sroka@...> wrote:

              > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...<mcoon1961%40gmail.com>>
              > wrote:
              > > In our sprints I have a QA person manually testing bits of functionality
              > as
              > > they are delivered; This generates defects as we go and we've been adding
              > a
              > > task to the Sprint backlog for each defect. I'm not completely satisfied
              > > with this approach, so I am asking what other, more experienced folks are
              > > doing with defects.
              >
              > I heard a description of how Google does this that I really liked.
              > Basically, they have a concept of "Certify in a Day". The idea is that
              > the product needs to be releasable at any time. So, if you find a bug
              > you need to fix the bug the same day or you won't be able to "Certify
              > in a Day." Of course, that is the goal and by their own admission they
              > don't always hit it and some product teams aren't anywhere close.
              > Still, it is a good way of keeping the fix close to the feature. Also,
              > Google depends heavily on automated tests and continuous builds to
              > enable this kind of rapidity.
              >
              >
              >



              --
              http://mikeonitstuff.net/ New Blog
              http://mikeonitstuff.com/ Old Blog
              http://mikeonbikes.blogspot.com/


              [Non-text portions of this message have been removed]
            • Olof Bjarnason
              ... So you have a communication issue?
              Message 6 of 18 , Jan 2, 2009
              • 0 Attachment
                2009/1/2 Mike Coon <mcoon1961@...>:
                > Yeah, I've been having a hard time getting traction on both test and
                > build/deploy automation; which seems odd given that we're all in the
                > automation business. Dev managers start with the - I think incorrect -
                > assumption that business people can't/don't/won't see the value of these
                > "infrastructure" activities.
                > The automation is the key for our situation because if I can take the time
                > and ambiguity of out deployment and testing, some other thing can become the
                > new bottleneck.
                >
                > One thing that happens now is we get defects from UAT folks that have not
                > gotten used to the idea that their defect should include steps, expected,
                > and actual results. So we spend a lot of time figuring out what they are
                > reporting which often turns out to be a feature request, training issue, or
                > user configuration setting - all of which do need to be identified and
                > resolved but currently take too long to understand, categorize, and address.

                So you have a communication issue?

                >
                > Thanks, Mike
                >
                > On Fri, Jan 2, 2009 at 1:33 PM, Adam Sroka <adam.sroka@...> wrote:
                >
                >> On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
                >> <mcoon1961@...<mcoon1961%40gmail.com>>
                >
                >> wrote:
                >> > In our sprints I have a QA person manually testing bits of functionality
                >> as
                >> > they are delivered; This generates defects as we go and we've been
                >> > adding
                >> a
                >> > task to the Sprint backlog for each defect. I'm not completely satisfied
                >> > with this approach, so I am asking what other, more experienced folks
                >> > are
                >> > doing with defects.
                >>
                >> I heard a description of how Google does this that I really liked.
                >> Basically, they have a concept of "Certify in a Day". The idea is that
                >> the product needs to be releasable at any time. So, if you find a bug
                >> you need to fix the bug the same day or you won't be able to "Certify
                >> in a Day." Of course, that is the goal and by their own admission they
                >> don't always hit it and some product teams aren't anywhere close.
                >> Still, it is a good way of keeping the fix close to the feature. Also,
                >> Google depends heavily on automated tests and continuous builds to
                >> enable this kind of rapidity.
                >>
                >>
                >>
                >
                > --
                > http://mikeonitstuff.net/ New Blog
                > http://mikeonitstuff.com/ Old Blog
                > http://mikeonbikes.blogspot.com/
                >
                > [Non-text portions of this message have been removed]
                >
                >
              • Charlie Poole
                Hi Mike, ... Leaving aside the importance of automated testing - and that s a pretty big thing to leave aside - I suggest that you not schedule any stories
                Message 7 of 18 , Jan 2, 2009
                • 0 Attachment
                  Hi Mike,

                  > Yes I agree with automated testing. For a variety of poor
                  > excuses we don't have much yet. I am talking about the churn
                  > of defect resolution where it seems we have a lot of
                  > stop/start as defects are opened, evaluated, fixed, and retested.

                  Leaving aside the importance of automated testing - and that's
                  a pretty big thing to leave aside - I suggest that you not
                  schedule any stories unless they have "acceptance tests"

                  In this context, I'm referring to a simple note on the back of
                  the story card that says how the story will be verified. If
                  it won't fit there, make it a reference to something else, but
                  be sure that something else isn't a full-fledged QA test plan.

                  All we are looking for at this stage is to know (for example)
                  whether certain exception cases are part of this story or are
                  intended to be dealt with in a later story. The text-based
                  acceptance test is an agreement with the customer (or QA
                  person) that evaluation of the story will be limited to
                  certain things. Without that, any story can grow to cover
                  an unbounded set of requirements or assumptions.

                  A further benefit of this simple step is that it gives you
                  something that can be automated if you decide to make time
                  - I won't say if you have the time - to do it.

                  Charlie

                  > I may use our unpleasant defect experience to make a better
                  > case to the dev managers for unit and functional testing. At
                  > this point they agree that it would be good but argue they
                  > don't have time. Predictably, we spend more time on defects
                  > than we would spend writing tests as we go.
                  >
                  > Most of our developers are not familiar with TDD at this point.
                  >
                  > Mike
                  >
                  > On Fri, Jan 2, 2009 at 12:09 PM, Keith Ray
                  > <keith.ray@...> wrote:
                  >
                  > > correctIon: "... these tests would have to PASS for the task to be
                  > >
                  > > acceptable at the end of the sprint."
                  > >
                  > > On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray
                  > > <keith.ray@...<keith.ray%40gmail.com>>
                  > > wrote:
                  > > > Well, I'm pretty old. I would recommend creating automated tests
                  > > > that fail because of the defect, and these tests would
                  > have to fail
                  > > > for the task to be acceptable at the end of the sprint.
                  > > >
                  > > > Think about how the functionality accumulates with each sprint -
                  > > > there will be soon no time for manual re-testing of previously
                  > > > implemented functionality. Without test automation, there
                  > could be
                  > > > undetected regressions.
                  > > >
                  > > > Some dialogs with the tester, development team and product owners
                  > > > may result in a "defect" being classified as a new
                  > backlog item, to
                  > > > be implemented a later sprint. This is particularly relevant for
                  > > > "missing" functionality.
                  > > >
                  > > >
                  > > > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
                  > > > <mcoon1961@...<mcoon1961%40gmail.com>>
                  > > wrote:
                  > > >> In our sprints I have a QA person manually testing bits of
                  > > >> functionality
                  > > as
                  > > >> they are delivered; This generates defects as we go and
                  > we've been
                  > > adding a
                  > > >> task to the Sprint backlog for each defect. I'm not completely
                  > > >> satisfied with this approach, so I am asking what other, more
                  > > >> experienced folks
                  > > are
                  > > >> doing with defects.
                  > > >> Thanks,
                  > > >>
                  > > >> --
                  > > >> http://mikeonitstuff.net/ New Blog
                  > > >> http://mikeonitstuff.com/ Old Blog
                  > > >> http://mikeonbikes.blogspot.com/
                  > > >>
                  > > >>
                  > > >> [Non-text portions of this message have been removed]
                  > > >>
                  > > >>
                  > > >> ------------------------------------
                  > > >>
                  > > >> To Post a message, send it to:
                  > > >> extremeprogramming@...<extremeprogramming%40eGroups.com>
                  > > >>
                  > > >> To Unsubscribe, send a blank message to:
                  > >
                  > extremeprogramming-unsubscribe@...<extremeprogramming-unsubscr
                  > > ibe%40eGroups.com>
                  > > >>
                  > > >> ad-free courtesy of objectmentor.comYahoo! Groups Links
                  > > >>
                  > > >>
                  > > >>
                  > > >>
                  > > >
                  > > >
                  > > >
                  > > > --
                  > > > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
                  > > > http://industriallogic.com 866-540-8336 (toll free)
                  > Groove with our
                  > > > Agile Greatest Hits:
                  > > http://www.industriallogic.com/elearning/
                  > > > http://agilesolutionspace.blogspot.com/
                  > > >
                  > >
                  > > --
                  > > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
                  > > http://industriallogic.com 866-540-8336 (toll free) Groove with our
                  > > Agile Greatest Hits:
                  > > http://www.industriallogic.com/elearning/
                  > > http://agilesolutionspace.blogspot.com/
                  > >
                  > >
                  > >
                  >
                  >
                  >
                  > --
                  > http://mikeonitstuff.net/ New Blog
                  > http://mikeonitstuff.com/ Old Blog
                  > http://mikeonbikes.blogspot.com/
                  >
                  >
                  > [Non-text portions of this message have been removed]
                  >
                  >
                  > ------------------------------------
                  >
                  > To Post a message, send it to: extremeprogramming@...
                  >
                  > To Unsubscribe, send a blank message to:
                  > extremeprogramming-unsubscribe@...
                  >
                  > ad-free courtesy of objectmentor.comYahoo! Groups Links
                  >
                  >
                  >
                  >
                • Charlie Poole
                  Hi Curt, ... I have to share my own favorite funny/sad excuse for not having acceptance tests... We don t really know what s wanted, so we can t write a test
                  Message 8 of 18 , Jan 2, 2009
                  • 0 Attachment
                    Hi Curt,

                    > Ahh, the old 'we dont' have time' argument, it cracks me up
                    > at this point. They don't have time because they or the
                    > organization is measuring the wrong metrics of productivity
                    > and/or lean/agile is not in practice at any higher level than
                    > the development teams (if there).

                    I have to share my own favorite funny/sad excuse for not
                    having acceptance tests...

                    "We don't really know what's wanted, so we can't write a test"

                    This was stated to me by a programmer who was busily writing
                    the code for the story he didn't understand.

                    Charlie
                  • Mike Coon
                    Don t almost all problems boil down to communication? Yes, we have a communication issue, exacerbated by geographic dispersion. Also, UAT is new to this
                    Message 9 of 18 , Jan 2, 2009
                    • 0 Attachment
                      Don't almost all problems boil down to communication? Yes, we have a
                      communication issue, exacerbated by geographic dispersion. Also, UAT is new
                      to this company and there is controversy over how it should be done, as well
                      as some weird thinking that test cases should not be shared amongst groups.

                      On Fri, Jan 2, 2009 at 1:56 PM, Olof Bjarnason <olof.bjarnason@...>wrote:

                      > 2009/1/2 Mike Coon <mcoon1961@... <mcoon1961%40gmail.com>>:
                      >
                      > > Yeah, I've been having a hard time getting traction on both test and
                      > > build/deploy automation; which seems odd given that we're all in the
                      > > automation business. Dev managers start with the - I think incorrect -
                      > > assumption that business people can't/don't/won't see the value of these
                      > > "infrastructure" activities.
                      > > The automation is the key for our situation because if I can take the
                      > time
                      > > and ambiguity of out deployment and testing, some other thing can become
                      > the
                      > > new bottleneck.
                      > >
                      > > One thing that happens now is we get defects from UAT folks that have not
                      > > gotten used to the idea that their defect should include steps, expected,
                      > > and actual results. So we spend a lot of time figuring out what they are
                      > > reporting which often turns out to be a feature request, training issue,
                      > or
                      > > user configuration setting - all of which do need to be identified and
                      > > resolved but currently take too long to understand, categorize, and
                      > address.
                      >
                      > So you have a communication issue?
                      >
                      > >
                      > > Thanks, Mike
                      > >
                      > > On Fri, Jan 2, 2009 at 1:33 PM, Adam Sroka <adam.sroka@...<adam.sroka%40gmail.com>>
                      > wrote:
                      > >
                      > >> On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
                      > >> <mcoon1961@... <mcoon1961%40gmail.com><mcoon1961%40gmail.com>>
                      > >
                      > >> wrote:
                      > >> > In our sprints I have a QA person manually testing bits of
                      > functionality
                      > >> as
                      > >> > they are delivered; This generates defects as we go and we've been
                      > >> > adding
                      > >> a
                      > >> > task to the Sprint backlog for each defect. I'm not completely
                      > satisfied
                      > >> > with this approach, so I am asking what other, more experienced folks
                      > >> > are
                      > >> > doing with defects.
                      > >>
                      > >> I heard a description of how Google does this that I really liked.
                      > >> Basically, they have a concept of "Certify in a Day". The idea is that
                      > >> the product needs to be releasable at any time. So, if you find a bug
                      > >> you need to fix the bug the same day or you won't be able to "Certify
                      > >> in a Day." Of course, that is the goal and by their own admission they
                      > >> don't always hit it and some product teams aren't anywhere close.
                      > >> Still, it is a good way of keeping the fix close to the feature. Also,
                      > >> Google depends heavily on automated tests and continuous builds to
                      > >> enable this kind of rapidity.
                      > >>
                      > >>
                      > >>
                      > >
                      > > --
                      > > http://mikeonitstuff.net/ New Blog
                      > > http://mikeonitstuff.com/ Old Blog
                      > > http://mikeonbikes.blogspot.com/
                      > >
                      > > [Non-text portions of this message have been removed]
                      > >
                      > >
                      >
                      >
                      >



                      --
                      http://mikeonitstuff.net/ New Blog
                      http://mikeonitstuff.com/ Old Blog
                      http://mikeonbikes.blogspot.com/


                      [Non-text portions of this message have been removed]
                    • Steven Gordon
                      On Fri, Jan 2, 2009 at 1:06 PM, Charlie Poole ... Somewhat better is if the acceptance test will not fit on the back of the card, slice the story into
                      Message 10 of 18 , Jan 2, 2009
                      • 0 Attachment
                        On Fri, Jan 2, 2009 at 1:06 PM, Charlie Poole
                        <charlie@...> wrote:
                        > Hi Mike,
                        >
                        >> Yes I agree with automated testing. For a variety of poor
                        >> excuses we don't have much yet. I am talking about the churn
                        >> of defect resolution where it seems we have a lot of
                        >> stop/start as defects are opened, evaluated, fixed, and retested.
                        >
                        > Leaving aside the importance of automated testing - and that's
                        > a pretty big thing to leave aside - I suggest that you not
                        > schedule any stories unless they have "acceptance tests"
                        >
                        > In this context, I'm referring to a simple note on the back of
                        > the story card that says how the story will be verified. If
                        > it won't fit there, make it a reference to something else, but
                        > be sure that something else isn't a full-fledged QA test plan.

                        Somewhat better is "if the acceptance test will not fit on the back of
                        the card, slice the story into substories each of which can have fit
                        its acceptance test on the back of a card".
                        In other words, make a story out of each individual acceptance test.

                        >
                        > All we are looking for at this stage is to know (for example)
                        > whether certain exception cases are part of this story or are
                        > intended to be dealt with in a later story. The text-based
                        > acceptance test is an agreement with the customer (or QA
                        > person) that evaluation of the story will be limited to
                        > certain things. Without that, any story can grow to cover
                        > an unbounded set of requirements or assumptions.
                        >
                        > A further benefit of this simple step is that it gives you
                        > something that can be automated if you decide to make time
                        > - I won't say if you have the time - to do it.
                        >
                        > Charlie
                        >
                      • Ron Jeffries
                        ... Are you doing XP? Do you have a customer? What happens if you simply ignore alleged defects without info? What are the lines of power on the project? Ron
                        Message 11 of 18 , Jan 2, 2009
                        • 0 Attachment
                          Hello, Mike. On Friday, January 2, 2009, at 1:50:03 PM, you wrote:

                          > One thing that happens now is we get defects from UAT folks that have not
                          > gotten used to the idea that their defect should include steps, expected,
                          > and actual results. So we spend a lot of time figuring out what they are
                          > reporting which often turns out to be a feature request, training issue, or
                          > user configuration setting - all of which do need to be identified and
                          > resolved but currently take too long to understand, categorize, and address.

                          Are you doing XP? Do you have a customer? What happens if you simply
                          ignore alleged defects without info? What are the lines of power on
                          the project?

                          Ron Jeffries
                          www.XProgramming.com
                          www.xprogramming.com/blog
                          Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                          of his head. It is, as far as he knows, the only way of coming downstairs,
                          but sometimes he feels that there really is another way, if only he could
                          stop bumping for a moment and think of it. And then he feels that perhaps
                          there isn't. -- A. A. Milne
                        • Charlie Poole
                          Hi Steven, ... Clearly. I was responding to (what I perceived to be) the level of the team in question. Slicing stories vertically is one of the harder
                          Message 12 of 18 , Jan 2, 2009
                          • 0 Attachment
                            Hi Steven,

                            > > Leaving aside the importance of automated testing - and that's a
                            > > pretty big thing to leave aside - I suggest that you not
                            > schedule any
                            > > stories unless they have "acceptance tests"
                            > >
                            > > In this context, I'm referring to a simple note on the back of the
                            > > story card that says how the story will be verified. If it
                            > won't fit
                            > > there, make it a reference to something else, but be sure that
                            > > something else isn't a full-fledged QA test plan.
                            >
                            > Somewhat better is "if the acceptance test will not fit on
                            > the back of the card, slice the story into substories each of
                            > which can have fit its acceptance test on the back of a card".
                            > In other words, make a story out of each individual acceptance test.

                            Clearly. I was responding to (what I perceived to be) the level
                            of the team in question. Slicing stories "vertically" is one of
                            the harder things for teams to learn.

                            But you're right, and that would be the next thing I'd push
                            toward - probably even before automation of the big tests.

                            Charlie
                          • George Dinwiddie
                            ... Mike, leaving aside the question of regression test automation, it sounds like you may doing the QA testing post iteration. On teams I ve worked with,
                            Message 13 of 18 , Jan 2, 2009
                            • 0 Attachment
                              Mike Coon wrote:
                              > In our sprints I have a QA person manually testing bits of functionality as
                              > they are delivered; This generates defects as we go and we've been adding a
                              > task to the Sprint backlog for each defect. I'm not completely satisfied
                              > with this approach, so I am asking what other, more experienced folks are
                              > doing with defects.

                              Mike, leaving aside the question of regression test automation, it
                              sounds like you may doing the QA testing post iteration. On teams I've
                              worked with, even when the test automation was weak, having the stories
                              tested within the iteration has proved to be a big help. When a defect
                              is found, it's just fixed--not added to the backlog. The story isn't
                              done until the testers (and the PO) are satisfied with it.

                              - George

                              --
                              ----------------------------------------------------------------------
                              * George Dinwiddie * http://blog.gdinwiddie.com
                              Software Development http://www.idiacomputing.com
                              Consultant and Coach http://www.agilemaryland.org
                              ----------------------------------------------------------------------
                            • Mike Coon
                              Ron, We are not doing XP though some folks use some practices. We do have a customer; after thinking about my situation here and considering many of the great
                              Message 14 of 18 , Jan 5, 2009
                              • 0 Attachment
                                Ron,

                                We are not doing XP though some folks use some practices. We do have a
                                customer; after thinking about my situation here and considering many of the
                                great responses I plan to focus more on the Product Owner and product
                                backlog for the next quarter.

                                As far as ignoring crappy defects - we tried that, and the consultant
                                working for the dev director punished us with unending meetings and
                                complaints - now we are bouncing them immediately asking for the steps,
                                expected and actual results. This still yeilds complaints but immunizes my
                                folks from much criticism as it's hard to actually say that you want us to
                                waste time figuring out what you were trying to do and what you thought
                                would happen.

                                Lines of power - I am still doing agile adoption as a side gig to my day job
                                of QA manager. The good news is that i have more influence every day and
                                can usually get my way. The bad news is that I can usually get my way - so
                                it helps to be right :-) While I am in my thirtieth year of IT work I'm
                                still learning just like everybody...

                                Thanks for the responses.

                                Mike

                                On Fri, Jan 2, 2009 at 5:06 PM, Ron Jeffries
                                <ronjeffries@...>wrote:

                                > Hello, Mike. On Friday, January 2, 2009, at 1:50:03 PM, you wrote:
                                >
                                > > One thing that happens now is we get defects from UAT folks that have not
                                > > gotten used to the idea that their defect should include steps, expected,
                                > > and actual results. So we spend a lot of time figuring out what they are
                                > > reporting which often turns out to be a feature request, training issue,
                                > or
                                > > user configuration setting - all of which do need to be identified and
                                > > resolved but currently take too long to understand, categorize, and
                                > address.
                                >
                                > Are you doing XP? Do you have a customer? What happens if you simply
                                > ignore alleged defects without info? What are the lines of power on
                                > the project?
                                >
                                > Ron Jeffries
                                > www.XProgramming.com
                                > www.xprogramming.com/blog
                                > Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                                > of his head. It is, as far as he knows, the only way of coming downstairs,
                                > but sometimes he feels that there really is another way, if only he could
                                > stop bumping for a moment and think of it. And then he feels that perhaps
                                > there isn't. -- A. A. Milne
                                >
                                >
                                >



                                --
                                http://mikeonitstuff.net/ New Blog
                                http://mikeonitstuff.com/ Old Blog
                                http://mikeonbikes.blogspot.com/


                                [Non-text portions of this message have been removed]
                              • Chet Hendrickson
                                Hello Mike, The first step in an XP style defect resolution process is the creation of concrete acceptance criteria. What Brian Marick calls Examples. These
                                Message 15 of 18 , Jan 5, 2009
                                • 0 Attachment
                                  Hello Mike,

                                  The first step in an XP style defect resolution process is the creation of concrete acceptance criteria. What Brian Marick calls Examples. These must belong to the project's Customer, although exactly who creates them is subject to the skills and temperaments of those on the team.
                                  In any case, the developers working on a feature need to have unambiguous guidelines as to how the system will behave once the new feature is added. This works best if they are available early in the development of the feature, but must be in hand before the developers declare the feature done.

                                  Tools like FitNesse are often used for this purpose. We mostly use the one we hate the least at any given moment.

                                  Once the examples exists, any developer who does not make use of them is opening themselves to great ridicule. This goes a long way towards eliminating the 'I thought it was supposed to do this' sort of defect.

                                  This process is not perfect. Often the Customer will come back with 'Yes, that is was I asked for, but this is really what I want'. That case is resolved by the addition of a new sort and its corresponding set of examples.

                                  chet

                                  Monday, January 5, 2009, 8:08:46 AM, you wrote:


                                  Ron,

                                  We are not doing XP though some folks use some practices. We do have a
                                  customer; after thinking about my situation here and considering many of the
                                  great responses I plan to focus more on the Product Owner and product
                                  backlog for the next quarter.

                                  As far as ignoring crappy defects - we tried that, and the consultant
                                  working for the dev director punished us with unending meetings and
                                  complaints - now we are bouncing them immediately asking for the steps,
                                  expected and actual results. This still yeilds complaints but immunizes my
                                  folks from much criticism as it's hard to actually say that you want us to
                                  waste time figuring out what you were trying to do and what you thought
                                  would happen.

                                  Lines of power - I am still doing agile adoption as a side gig to my day job
                                  of QA manager. The good news is that i have more influence every day and
                                  can usually get my way. The bad news is that I can usually get my way - so
                                  it helps to be right :-) While I am in my thirtieth year of IT work I'm
                                  still learning just like everybody...

                                  Thanks for the responses.

                                  Mike

                                  On Fri, Jan 2, 2009 at 5:06 PM, Ron Jeffries
                                  <ronjeffries@...>wrote:

                                  > Hello, Mike. On Friday, January 2, 2009, at 1:50:03 PM, you wrote:
                                  >
                                  > > One thing that happens now is we get defects from UAT folks that have not
                                  > > gotten used to the idea that their defect should include steps, expected,
                                  > > and actual results. So we spend a lot of time figuring out what they are
                                  > > reporting which often turns out to be a feature request, training issue,
                                  > or
                                  > > user configuration setting - all of which do need to be identified and
                                  > > resolved but currently take too long to understand, categorize, and
                                  > address.
                                  >
                                  > Are you doing XP? Do you have a customer? What happens if you simply
                                  > ignore alleged defects without info? What are the lines of power on
                                  > the project?
                                  >
                                  > Ron Jeffries
                                  > www.XProgramming.com
                                  > www.xprogramming.com/blog
                                  > Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                                  > of his head. It is, as far as he knows, the only way of coming downstairs,
                                  > but sometimes he feels that there really is another way, if only he could
                                  > stop bumping for a moment and think of it. And then he feels that perhaps
                                  > there isn't. -- A. A. Milne
                                  >
                                  >
                                  >

                                  --
                                  http://mikeonitstuff.net/ New Blog
                                  http://mikeonitstuff.com/ Old Blog
                                  http://mikeonbikes.blogspot.com/

                                  [Non-text portions of this message have been removed]







                                  --
                                  Best regards,
                                  Chet mailto:lists@...

                                  [Non-text portions of this message have been removed]
                                • Tim Ottinger
                                  ... /me nods and weeps.
                                  Message 16 of 18 , Jan 6, 2009
                                  • 0 Attachment
                                    > From: Charlie Poole <charlie@...>
                                    > I have to share my own favorite funny/sad excuse for not
                                    > having acceptance tests...
                                    >
                                    > "We don't really know what's wanted, so we can't write a test"
                                    >
                                    > This was stated to me by a programmer who was busily writing
                                    > the code for the story he didn't understand.

                                    /me nods and weeps.
                                  Your message has been successfully submitted and would be delivered to recipients shortly.