Loading ...
Sorry, an error occurred while loading the content.

How do you handle defect resolution?

Expand Messages
  • Mike Coon
    In our sprints I have a QA person manually testing bits of functionality as they are delivered; This generates defects as we go and we ve been adding a task
    Message 1 of 18 , Jan 2, 2009
    • 0 Attachment
      In our sprints I have a QA person manually testing bits of functionality as
      they are delivered; This generates defects as we go and we've been adding a
      task to the Sprint backlog for each defect. I'm not completely satisfied
      with this approach, so I am asking what other, more experienced folks are
      doing with defects.
      Thanks,

      --
      http://mikeonitstuff.net/ New Blog
      http://mikeonitstuff.com/ Old Blog
      http://mikeonbikes.blogspot.com/


      [Non-text portions of this message have been removed]
    • Keith Ray
      Well, I m pretty old. I would recommend creating automated tests that fail because of the defect, and these tests would have to fail for the task to be
      Message 2 of 18 , Jan 2, 2009
      • 0 Attachment
        Well, I'm pretty old. I would recommend creating automated tests that
        fail because of the defect, and these tests would have to fail for the
        task to be acceptable at the end of the sprint.

        Think about how the functionality accumulates with each sprint - there
        will be soon no time for manual re-testing of previously implemented
        functionality. Without test automation, there could be undetected
        regressions.

        Some dialogs with the tester, development team and product owners may
        result in a "defect" being classified as a new backlog item, to be
        implemented a later sprint. This is particularly relevant for
        "missing" functionality.


        On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...> wrote:
        > In our sprints I have a QA person manually testing bits of functionality as
        > they are delivered; This generates defects as we go and we've been adding a
        > task to the Sprint backlog for each defect. I'm not completely satisfied
        > with this approach, so I am asking what other, more experienced folks are
        > doing with defects.
        > Thanks,
        >
        > --
        > http://mikeonitstuff.net/ New Blog
        > http://mikeonitstuff.com/ Old Blog
        > http://mikeonbikes.blogspot.com/
        >
        >
        > [Non-text portions of this message have been removed]
        >
        >
        > ------------------------------------
        >
        > To Post a message, send it to: extremeprogramming@...
        >
        > To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
        >
        > ad-free courtesy of objectmentor.comYahoo! Groups Links
        >
        >
        >
        >



        --
        C. Keith Ray, IXP Coach, Industrial Logic, Inc.
        http://industriallogic.com 866-540-8336 (toll free)
        Groove with our Agile Greatest Hits: http://www.industriallogic.com/elearning/
        http://agilesolutionspace.blogspot.com/
      • Keith Ray
        correctIon: ... these tests would have to PASS for the task to be acceptable at the end of the sprint. ... -- C. Keith Ray, IXP Coach, Industrial Logic,
        Message 3 of 18 , Jan 2, 2009
        • 0 Attachment
          correctIon: "... these tests would have to PASS for the task to be
          acceptable at the end of the sprint."

          On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray <keith.ray@...> wrote:
          > Well, I'm pretty old. I would recommend creating automated tests that
          > fail because of the defect, and these tests would have to fail for the
          > task to be acceptable at the end of the sprint.
          >
          > Think about how the functionality accumulates with each sprint - there
          > will be soon no time for manual re-testing of previously implemented
          > functionality. Without test automation, there could be undetected
          > regressions.
          >
          > Some dialogs with the tester, development team and product owners may
          > result in a "defect" being classified as a new backlog item, to be
          > implemented a later sprint. This is particularly relevant for
          > "missing" functionality.
          >
          >
          > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...> wrote:
          >> In our sprints I have a QA person manually testing bits of functionality as
          >> they are delivered; This generates defects as we go and we've been adding a
          >> task to the Sprint backlog for each defect. I'm not completely satisfied
          >> with this approach, so I am asking what other, more experienced folks are
          >> doing with defects.
          >> Thanks,
          >>
          >> --
          >> http://mikeonitstuff.net/ New Blog
          >> http://mikeonitstuff.com/ Old Blog
          >> http://mikeonbikes.blogspot.com/
          >>
          >>
          >> [Non-text portions of this message have been removed]
          >>
          >>
          >> ------------------------------------
          >>
          >> To Post a message, send it to: extremeprogramming@...
          >>
          >> To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
          >>
          >> ad-free courtesy of objectmentor.comYahoo! Groups Links
          >>
          >>
          >>
          >>
          >
          >
          >
          > --
          > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
          > http://industriallogic.com 866-540-8336 (toll free)
          > Groove with our Agile Greatest Hits: http://www.industriallogic.com/elearning/
          > http://agilesolutionspace.blogspot.com/
          >



          --
          C. Keith Ray, IXP Coach, Industrial Logic, Inc.
          http://industriallogic.com 866-540-8336 (toll free)
          Groove with our Agile Greatest Hits: http://www.industriallogic.com/elearning/
          http://agilesolutionspace.blogspot.com/
        • Mike Coon
          Thanks for the response - sometimes just talking about things brings clarity. Yes I agree with automated testing. For a variety of poor excuses we don t have
          Message 4 of 18 , Jan 2, 2009
          • 0 Attachment
            Thanks for the response - sometimes just talking about things brings
            clarity.

            Yes I agree with automated testing. For a variety of poor excuses we don't
            have much yet. I am talking about the churn of defect resolution where it
            seems we have a lot of stop/start as defects are opened, evaluated, fixed,
            and retested.
            I may use our unpleasant defect experience to make a better case to the dev
            managers for unit and functional testing. At this point they agree that it
            would be good but argue they don't have time. Predictably, we spend more
            time on defects than we would spend writing tests as we go.

            Most of our developers are not familiar with TDD at this point.

            Mike

            On Fri, Jan 2, 2009 at 12:09 PM, Keith Ray <keith.ray@...> wrote:

            > correctIon: "... these tests would have to PASS for the task to be
            >
            > acceptable at the end of the sprint."
            >
            > On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray <keith.ray@...<keith.ray%40gmail.com>>
            > wrote:
            > > Well, I'm pretty old. I would recommend creating automated tests that
            > > fail because of the defect, and these tests would have to fail for the
            > > task to be acceptable at the end of the sprint.
            > >
            > > Think about how the functionality accumulates with each sprint - there
            > > will be soon no time for manual re-testing of previously implemented
            > > functionality. Without test automation, there could be undetected
            > > regressions.
            > >
            > > Some dialogs with the tester, development team and product owners may
            > > result in a "defect" being classified as a new backlog item, to be
            > > implemented a later sprint. This is particularly relevant for
            > > "missing" functionality.
            > >
            > >
            > > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...<mcoon1961%40gmail.com>>
            > wrote:
            > >> In our sprints I have a QA person manually testing bits of functionality
            > as
            > >> they are delivered; This generates defects as we go and we've been
            > adding a
            > >> task to the Sprint backlog for each defect. I'm not completely satisfied
            > >> with this approach, so I am asking what other, more experienced folks
            > are
            > >> doing with defects.
            > >> Thanks,
            > >>
            > >> --
            > >> http://mikeonitstuff.net/ New Blog
            > >> http://mikeonitstuff.com/ Old Blog
            > >> http://mikeonbikes.blogspot.com/
            > >>
            > >>
            > >> [Non-text portions of this message have been removed]
            > >>
            > >>
            > >> ------------------------------------
            > >>
            > >> To Post a message, send it to: extremeprogramming@...<extremeprogramming%40eGroups.com>
            > >>
            > >> To Unsubscribe, send a blank message to:
            > extremeprogramming-unsubscribe@...<extremeprogramming-unsubscribe%40eGroups.com>
            > >>
            > >> ad-free courtesy of objectmentor.comYahoo! Groups Links
            > >>
            > >>
            > >>
            > >>
            > >
            > >
            > >
            > > --
            > > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
            > > http://industriallogic.com 866-540-8336 (toll free)
            > > Groove with our Agile Greatest Hits:
            > http://www.industriallogic.com/elearning/
            > > http://agilesolutionspace.blogspot.com/
            > >
            >
            > --
            > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
            > http://industriallogic.com 866-540-8336 (toll free)
            > Groove with our Agile Greatest Hits:
            > http://www.industriallogic.com/elearning/
            > http://agilesolutionspace.blogspot.com/
            >
            >
            >



            --
            http://mikeonitstuff.net/ New Blog
            http://mikeonitstuff.com/ Old Blog
            http://mikeonbikes.blogspot.com/


            [Non-text portions of this message have been removed]
          • Curt Coulter
            This is what I do. QA testing is happening in real time with stories being closed. Defects that materially affect the functionality being delivered, if
            Message 5 of 18 , Jan 2, 2009
            • 0 Attachment
              This is what I do. QA testing is happening in real time with stories
              being closed. Defects that materially affect the functionality being
              delivered, if missed by the automated tests, cause the story to 'test
              fail' and we of course do not get the points for that story until it
              is completely test passed. Any defects are closed as part of the
              story, using the priority order as set in the sprint tasks. In this
              case, the developers are to write a (failing) automated unit test
              first, and then fix the bug.

              If the defect is something ancillary, the product owner(s) and I
              evaluate it as to whether it is required for the story in question, or
              can go on the backlog for the next sprint.

              Ahh, the old 'we dont' have time' argument, it cracks me up at this
              point. They don't have time because they or the organization is
              measuring the wrong metrics of productivity and/or lean/agile is not
              in practice at any higher level than the development teams (if there).
              In my experience, nothing will change until quality becomes the
              primary focus.

              /Curt

              On Fri, Jan 2, 2009 at 12:22 PM, Mike Coon <mcoon1961@...> wrote:
              > Thanks for the response - sometimes just talking about things brings
              > clarity.
              >
              > Yes I agree with automated testing. For a variety of poor excuses we don't
              > have much yet. I am talking about the churn of defect resolution where it
              > seems we have a lot of stop/start as defects are opened, evaluated, fixed,
              > and retested.
              > I may use our unpleasant defect experience to make a better case to the dev
              > managers for unit and functional testing. At this point they agree that it
              > would be good but argue they don't have time. Predictably, we spend more
              > time on defects than we would spend writing tests as we go.
              >
              > Most of our developers are not familiar with TDD at this point.
              >
              > Mike
              >
              > On Fri, Jan 2, 2009 at 12:09 PM, Keith Ray <keith.ray@...> wrote:
              >
              >> correctIon: "... these tests would have to PASS for the task to be
              >>
              >> acceptable at the end of the sprint."
              >>
              >> On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray
              >> <keith.ray@...<keith.ray%40gmail.com>>
              >> wrote:
              >> > Well, I'm pretty old. I would recommend creating automated tests that
              >> > fail because of the defect, and these tests would have to fail for the
              >> > task to be acceptable at the end of the sprint.
              >> >
              >> > Think about how the functionality accumulates with each sprint - there
              >> > will be soon no time for manual re-testing of previously implemented
              >> > functionality. Without test automation, there could be undetected
              >> > regressions.
              >> >
              >> > Some dialogs with the tester, development team and product owners may
              >> > result in a "defect" being classified as a new backlog item, to be
              >> > implemented a later sprint. This is particularly relevant for
              >> > "missing" functionality.
              >> >
              >> >
              >> > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
              >> > <mcoon1961@...<mcoon1961%40gmail.com>>
              >> wrote:
              >> >> In our sprints I have a QA person manually testing bits of
              >> >> functionality
              >> as
              >> >> they are delivered; This generates defects as we go and we've been
              >> adding a
              >> >> task to the Sprint backlog for each defect. I'm not completely
              >> >> satisfied
              >> >> with this approach, so I am asking what other, more experienced folks
              >> are
              >> >> doing with defects.
              >> >> Thanks,
              >> >>
              >> >> --
              >> >> http://mikeonitstuff.net/ New Blog
              >> >> http://mikeonitstuff.com/ Old Blog
              >> >> http://mikeonbikes.blogspot.com/
              >> >>
              >> >>
              >> >> [Non-text portions of this message have been removed]
              >> >>
              >> >>
              >> >> ------------------------------------
              >> >>
              >> >> To Post a message, send it to:
              >> >> extremeprogramming@...<extremeprogramming%40eGroups.com>
              >> >>
              >> >> To Unsubscribe, send a blank message to:
              >>
              >> extremeprogramming-unsubscribe@...<extremeprogramming-unsubscribe%40eGroups.com>
              >> >>
              >> >> ad-free courtesy of objectmentor.comYahoo! Groups Links
              >> >>
              >> >>
              >> >>
              >> >>
              >> >
              >> >
              >> >
              >> > --
              >> > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
              >> > http://industriallogic.com 866-540-8336 (toll free)
              >> > Groove with our Agile Greatest Hits:
              >> http://www.industriallogic.com/elearning/
              >> > http://agilesolutionspace.blogspot.com/
              >> >
              >>
              >> --
              >> C. Keith Ray, IXP Coach, Industrial Logic, Inc.
              >> http://industriallogic.com 866-540-8336 (toll free)
              >> Groove with our Agile Greatest Hits:
              >> http://www.industriallogic.com/elearning/
              >> http://agilesolutionspace.blogspot.com/
              >>
              >>
              >>
              >
              > --
              > http://mikeonitstuff.net/ New Blog
              > http://mikeonitstuff.com/ Old Blog
              > http://mikeonbikes.blogspot.com/
              >
              > [Non-text portions of this message have been removed]
              >
              >
            • Adam Sroka
              ... I heard a description of how Google does this that I really liked. Basically, they have a concept of Certify in a Day . The idea is that the product needs
              Message 6 of 18 , Jan 2, 2009
              • 0 Attachment
                On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...> wrote:
                > In our sprints I have a QA person manually testing bits of functionality as
                > they are delivered; This generates defects as we go and we've been adding a
                > task to the Sprint backlog for each defect. I'm not completely satisfied
                > with this approach, so I am asking what other, more experienced folks are
                > doing with defects.

                I heard a description of how Google does this that I really liked.
                Basically, they have a concept of "Certify in a Day". The idea is that
                the product needs to be releasable at any time. So, if you find a bug
                you need to fix the bug the same day or you won't be able to "Certify
                in a Day." Of course, that is the goal and by their own admission they
                don't always hit it and some product teams aren't anywhere close.
                Still, it is a good way of keeping the fix close to the feature. Also,
                Google depends heavily on automated tests and continuous builds to
                enable this kind of rapidity.
              • Mike Coon
                Yeah, I ve been having a hard time getting traction on both test and build/deploy automation; which seems odd given that we re all in the automation business.
                Message 7 of 18 , Jan 2, 2009
                • 0 Attachment
                  Yeah, I've been having a hard time getting traction on both test and
                  build/deploy automation; which seems odd given that we're all in the
                  automation business. Dev managers start with the - I think incorrect -
                  assumption that business people can't/don't/won't see the value of these
                  "infrastructure" activities.
                  The automation is the key for our situation because if I can take the time
                  and ambiguity of out deployment and testing, some other thing can become the
                  new bottleneck.

                  One thing that happens now is we get defects from UAT folks that have not
                  gotten used to the idea that their defect should include steps, expected,
                  and actual results. So we spend a lot of time figuring out what they are
                  reporting which often turns out to be a feature request, training issue, or
                  user configuration setting - all of which do need to be identified and
                  resolved but currently take too long to understand, categorize, and address.

                  Thanks, Mike

                  On Fri, Jan 2, 2009 at 1:33 PM, Adam Sroka <adam.sroka@...> wrote:

                  > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon <mcoon1961@...<mcoon1961%40gmail.com>>
                  > wrote:
                  > > In our sprints I have a QA person manually testing bits of functionality
                  > as
                  > > they are delivered; This generates defects as we go and we've been adding
                  > a
                  > > task to the Sprint backlog for each defect. I'm not completely satisfied
                  > > with this approach, so I am asking what other, more experienced folks are
                  > > doing with defects.
                  >
                  > I heard a description of how Google does this that I really liked.
                  > Basically, they have a concept of "Certify in a Day". The idea is that
                  > the product needs to be releasable at any time. So, if you find a bug
                  > you need to fix the bug the same day or you won't be able to "Certify
                  > in a Day." Of course, that is the goal and by their own admission they
                  > don't always hit it and some product teams aren't anywhere close.
                  > Still, it is a good way of keeping the fix close to the feature. Also,
                  > Google depends heavily on automated tests and continuous builds to
                  > enable this kind of rapidity.
                  >
                  >
                  >



                  --
                  http://mikeonitstuff.net/ New Blog
                  http://mikeonitstuff.com/ Old Blog
                  http://mikeonbikes.blogspot.com/


                  [Non-text portions of this message have been removed]
                • Olof Bjarnason
                  ... So you have a communication issue?
                  Message 8 of 18 , Jan 2, 2009
                  • 0 Attachment
                    2009/1/2 Mike Coon <mcoon1961@...>:
                    > Yeah, I've been having a hard time getting traction on both test and
                    > build/deploy automation; which seems odd given that we're all in the
                    > automation business. Dev managers start with the - I think incorrect -
                    > assumption that business people can't/don't/won't see the value of these
                    > "infrastructure" activities.
                    > The automation is the key for our situation because if I can take the time
                    > and ambiguity of out deployment and testing, some other thing can become the
                    > new bottleneck.
                    >
                    > One thing that happens now is we get defects from UAT folks that have not
                    > gotten used to the idea that their defect should include steps, expected,
                    > and actual results. So we spend a lot of time figuring out what they are
                    > reporting which often turns out to be a feature request, training issue, or
                    > user configuration setting - all of which do need to be identified and
                    > resolved but currently take too long to understand, categorize, and address.

                    So you have a communication issue?

                    >
                    > Thanks, Mike
                    >
                    > On Fri, Jan 2, 2009 at 1:33 PM, Adam Sroka <adam.sroka@...> wrote:
                    >
                    >> On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
                    >> <mcoon1961@...<mcoon1961%40gmail.com>>
                    >
                    >> wrote:
                    >> > In our sprints I have a QA person manually testing bits of functionality
                    >> as
                    >> > they are delivered; This generates defects as we go and we've been
                    >> > adding
                    >> a
                    >> > task to the Sprint backlog for each defect. I'm not completely satisfied
                    >> > with this approach, so I am asking what other, more experienced folks
                    >> > are
                    >> > doing with defects.
                    >>
                    >> I heard a description of how Google does this that I really liked.
                    >> Basically, they have a concept of "Certify in a Day". The idea is that
                    >> the product needs to be releasable at any time. So, if you find a bug
                    >> you need to fix the bug the same day or you won't be able to "Certify
                    >> in a Day." Of course, that is the goal and by their own admission they
                    >> don't always hit it and some product teams aren't anywhere close.
                    >> Still, it is a good way of keeping the fix close to the feature. Also,
                    >> Google depends heavily on automated tests and continuous builds to
                    >> enable this kind of rapidity.
                    >>
                    >>
                    >>
                    >
                    > --
                    > http://mikeonitstuff.net/ New Blog
                    > http://mikeonitstuff.com/ Old Blog
                    > http://mikeonbikes.blogspot.com/
                    >
                    > [Non-text portions of this message have been removed]
                    >
                    >
                  • Charlie Poole
                    Hi Mike, ... Leaving aside the importance of automated testing - and that s a pretty big thing to leave aside - I suggest that you not schedule any stories
                    Message 9 of 18 , Jan 2, 2009
                    • 0 Attachment
                      Hi Mike,

                      > Yes I agree with automated testing. For a variety of poor
                      > excuses we don't have much yet. I am talking about the churn
                      > of defect resolution where it seems we have a lot of
                      > stop/start as defects are opened, evaluated, fixed, and retested.

                      Leaving aside the importance of automated testing - and that's
                      a pretty big thing to leave aside - I suggest that you not
                      schedule any stories unless they have "acceptance tests"

                      In this context, I'm referring to a simple note on the back of
                      the story card that says how the story will be verified. If
                      it won't fit there, make it a reference to something else, but
                      be sure that something else isn't a full-fledged QA test plan.

                      All we are looking for at this stage is to know (for example)
                      whether certain exception cases are part of this story or are
                      intended to be dealt with in a later story. The text-based
                      acceptance test is an agreement with the customer (or QA
                      person) that evaluation of the story will be limited to
                      certain things. Without that, any story can grow to cover
                      an unbounded set of requirements or assumptions.

                      A further benefit of this simple step is that it gives you
                      something that can be automated if you decide to make time
                      - I won't say if you have the time - to do it.

                      Charlie

                      > I may use our unpleasant defect experience to make a better
                      > case to the dev managers for unit and functional testing. At
                      > this point they agree that it would be good but argue they
                      > don't have time. Predictably, we spend more time on defects
                      > than we would spend writing tests as we go.
                      >
                      > Most of our developers are not familiar with TDD at this point.
                      >
                      > Mike
                      >
                      > On Fri, Jan 2, 2009 at 12:09 PM, Keith Ray
                      > <keith.ray@...> wrote:
                      >
                      > > correctIon: "... these tests would have to PASS for the task to be
                      > >
                      > > acceptable at the end of the sprint."
                      > >
                      > > On Fri, Jan 2, 2009 at 9:08 AM, Keith Ray
                      > > <keith.ray@...<keith.ray%40gmail.com>>
                      > > wrote:
                      > > > Well, I'm pretty old. I would recommend creating automated tests
                      > > > that fail because of the defect, and these tests would
                      > have to fail
                      > > > for the task to be acceptable at the end of the sprint.
                      > > >
                      > > > Think about how the functionality accumulates with each sprint -
                      > > > there will be soon no time for manual re-testing of previously
                      > > > implemented functionality. Without test automation, there
                      > could be
                      > > > undetected regressions.
                      > > >
                      > > > Some dialogs with the tester, development team and product owners
                      > > > may result in a "defect" being classified as a new
                      > backlog item, to
                      > > > be implemented a later sprint. This is particularly relevant for
                      > > > "missing" functionality.
                      > > >
                      > > >
                      > > > On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
                      > > > <mcoon1961@...<mcoon1961%40gmail.com>>
                      > > wrote:
                      > > >> In our sprints I have a QA person manually testing bits of
                      > > >> functionality
                      > > as
                      > > >> they are delivered; This generates defects as we go and
                      > we've been
                      > > adding a
                      > > >> task to the Sprint backlog for each defect. I'm not completely
                      > > >> satisfied with this approach, so I am asking what other, more
                      > > >> experienced folks
                      > > are
                      > > >> doing with defects.
                      > > >> Thanks,
                      > > >>
                      > > >> --
                      > > >> http://mikeonitstuff.net/ New Blog
                      > > >> http://mikeonitstuff.com/ Old Blog
                      > > >> http://mikeonbikes.blogspot.com/
                      > > >>
                      > > >>
                      > > >> [Non-text portions of this message have been removed]
                      > > >>
                      > > >>
                      > > >> ------------------------------------
                      > > >>
                      > > >> To Post a message, send it to:
                      > > >> extremeprogramming@...<extremeprogramming%40eGroups.com>
                      > > >>
                      > > >> To Unsubscribe, send a blank message to:
                      > >
                      > extremeprogramming-unsubscribe@...<extremeprogramming-unsubscr
                      > > ibe%40eGroups.com>
                      > > >>
                      > > >> ad-free courtesy of objectmentor.comYahoo! Groups Links
                      > > >>
                      > > >>
                      > > >>
                      > > >>
                      > > >
                      > > >
                      > > >
                      > > > --
                      > > > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
                      > > > http://industriallogic.com 866-540-8336 (toll free)
                      > Groove with our
                      > > > Agile Greatest Hits:
                      > > http://www.industriallogic.com/elearning/
                      > > > http://agilesolutionspace.blogspot.com/
                      > > >
                      > >
                      > > --
                      > > C. Keith Ray, IXP Coach, Industrial Logic, Inc.
                      > > http://industriallogic.com 866-540-8336 (toll free) Groove with our
                      > > Agile Greatest Hits:
                      > > http://www.industriallogic.com/elearning/
                      > > http://agilesolutionspace.blogspot.com/
                      > >
                      > >
                      > >
                      >
                      >
                      >
                      > --
                      > http://mikeonitstuff.net/ New Blog
                      > http://mikeonitstuff.com/ Old Blog
                      > http://mikeonbikes.blogspot.com/
                      >
                      >
                      > [Non-text portions of this message have been removed]
                      >
                      >
                      > ------------------------------------
                      >
                      > To Post a message, send it to: extremeprogramming@...
                      >
                      > To Unsubscribe, send a blank message to:
                      > extremeprogramming-unsubscribe@...
                      >
                      > ad-free courtesy of objectmentor.comYahoo! Groups Links
                      >
                      >
                      >
                      >
                    • Charlie Poole
                      Hi Curt, ... I have to share my own favorite funny/sad excuse for not having acceptance tests... We don t really know what s wanted, so we can t write a test
                      Message 10 of 18 , Jan 2, 2009
                      • 0 Attachment
                        Hi Curt,

                        > Ahh, the old 'we dont' have time' argument, it cracks me up
                        > at this point. They don't have time because they or the
                        > organization is measuring the wrong metrics of productivity
                        > and/or lean/agile is not in practice at any higher level than
                        > the development teams (if there).

                        I have to share my own favorite funny/sad excuse for not
                        having acceptance tests...

                        "We don't really know what's wanted, so we can't write a test"

                        This was stated to me by a programmer who was busily writing
                        the code for the story he didn't understand.

                        Charlie
                      • Mike Coon
                        Don t almost all problems boil down to communication? Yes, we have a communication issue, exacerbated by geographic dispersion. Also, UAT is new to this
                        Message 11 of 18 , Jan 2, 2009
                        • 0 Attachment
                          Don't almost all problems boil down to communication? Yes, we have a
                          communication issue, exacerbated by geographic dispersion. Also, UAT is new
                          to this company and there is controversy over how it should be done, as well
                          as some weird thinking that test cases should not be shared amongst groups.

                          On Fri, Jan 2, 2009 at 1:56 PM, Olof Bjarnason <olof.bjarnason@...>wrote:

                          > 2009/1/2 Mike Coon <mcoon1961@... <mcoon1961%40gmail.com>>:
                          >
                          > > Yeah, I've been having a hard time getting traction on both test and
                          > > build/deploy automation; which seems odd given that we're all in the
                          > > automation business. Dev managers start with the - I think incorrect -
                          > > assumption that business people can't/don't/won't see the value of these
                          > > "infrastructure" activities.
                          > > The automation is the key for our situation because if I can take the
                          > time
                          > > and ambiguity of out deployment and testing, some other thing can become
                          > the
                          > > new bottleneck.
                          > >
                          > > One thing that happens now is we get defects from UAT folks that have not
                          > > gotten used to the idea that their defect should include steps, expected,
                          > > and actual results. So we spend a lot of time figuring out what they are
                          > > reporting which often turns out to be a feature request, training issue,
                          > or
                          > > user configuration setting - all of which do need to be identified and
                          > > resolved but currently take too long to understand, categorize, and
                          > address.
                          >
                          > So you have a communication issue?
                          >
                          > >
                          > > Thanks, Mike
                          > >
                          > > On Fri, Jan 2, 2009 at 1:33 PM, Adam Sroka <adam.sroka@...<adam.sroka%40gmail.com>>
                          > wrote:
                          > >
                          > >> On Fri, Jan 2, 2009 at 8:59 AM, Mike Coon
                          > >> <mcoon1961@... <mcoon1961%40gmail.com><mcoon1961%40gmail.com>>
                          > >
                          > >> wrote:
                          > >> > In our sprints I have a QA person manually testing bits of
                          > functionality
                          > >> as
                          > >> > they are delivered; This generates defects as we go and we've been
                          > >> > adding
                          > >> a
                          > >> > task to the Sprint backlog for each defect. I'm not completely
                          > satisfied
                          > >> > with this approach, so I am asking what other, more experienced folks
                          > >> > are
                          > >> > doing with defects.
                          > >>
                          > >> I heard a description of how Google does this that I really liked.
                          > >> Basically, they have a concept of "Certify in a Day". The idea is that
                          > >> the product needs to be releasable at any time. So, if you find a bug
                          > >> you need to fix the bug the same day or you won't be able to "Certify
                          > >> in a Day." Of course, that is the goal and by their own admission they
                          > >> don't always hit it and some product teams aren't anywhere close.
                          > >> Still, it is a good way of keeping the fix close to the feature. Also,
                          > >> Google depends heavily on automated tests and continuous builds to
                          > >> enable this kind of rapidity.
                          > >>
                          > >>
                          > >>
                          > >
                          > > --
                          > > http://mikeonitstuff.net/ New Blog
                          > > http://mikeonitstuff.com/ Old Blog
                          > > http://mikeonbikes.blogspot.com/
                          > >
                          > > [Non-text portions of this message have been removed]
                          > >
                          > >
                          >
                          >
                          >



                          --
                          http://mikeonitstuff.net/ New Blog
                          http://mikeonitstuff.com/ Old Blog
                          http://mikeonbikes.blogspot.com/


                          [Non-text portions of this message have been removed]
                        • Steven Gordon
                          On Fri, Jan 2, 2009 at 1:06 PM, Charlie Poole ... Somewhat better is if the acceptance test will not fit on the back of the card, slice the story into
                          Message 12 of 18 , Jan 2, 2009
                          • 0 Attachment
                            On Fri, Jan 2, 2009 at 1:06 PM, Charlie Poole
                            <charlie@...> wrote:
                            > Hi Mike,
                            >
                            >> Yes I agree with automated testing. For a variety of poor
                            >> excuses we don't have much yet. I am talking about the churn
                            >> of defect resolution where it seems we have a lot of
                            >> stop/start as defects are opened, evaluated, fixed, and retested.
                            >
                            > Leaving aside the importance of automated testing - and that's
                            > a pretty big thing to leave aside - I suggest that you not
                            > schedule any stories unless they have "acceptance tests"
                            >
                            > In this context, I'm referring to a simple note on the back of
                            > the story card that says how the story will be verified. If
                            > it won't fit there, make it a reference to something else, but
                            > be sure that something else isn't a full-fledged QA test plan.

                            Somewhat better is "if the acceptance test will not fit on the back of
                            the card, slice the story into substories each of which can have fit
                            its acceptance test on the back of a card".
                            In other words, make a story out of each individual acceptance test.

                            >
                            > All we are looking for at this stage is to know (for example)
                            > whether certain exception cases are part of this story or are
                            > intended to be dealt with in a later story. The text-based
                            > acceptance test is an agreement with the customer (or QA
                            > person) that evaluation of the story will be limited to
                            > certain things. Without that, any story can grow to cover
                            > an unbounded set of requirements or assumptions.
                            >
                            > A further benefit of this simple step is that it gives you
                            > something that can be automated if you decide to make time
                            > - I won't say if you have the time - to do it.
                            >
                            > Charlie
                            >
                          • Ron Jeffries
                            ... Are you doing XP? Do you have a customer? What happens if you simply ignore alleged defects without info? What are the lines of power on the project? Ron
                            Message 13 of 18 , Jan 2, 2009
                            • 0 Attachment
                              Hello, Mike. On Friday, January 2, 2009, at 1:50:03 PM, you wrote:

                              > One thing that happens now is we get defects from UAT folks that have not
                              > gotten used to the idea that their defect should include steps, expected,
                              > and actual results. So we spend a lot of time figuring out what they are
                              > reporting which often turns out to be a feature request, training issue, or
                              > user configuration setting - all of which do need to be identified and
                              > resolved but currently take too long to understand, categorize, and address.

                              Are you doing XP? Do you have a customer? What happens if you simply
                              ignore alleged defects without info? What are the lines of power on
                              the project?

                              Ron Jeffries
                              www.XProgramming.com
                              www.xprogramming.com/blog
                              Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                              of his head. It is, as far as he knows, the only way of coming downstairs,
                              but sometimes he feels that there really is another way, if only he could
                              stop bumping for a moment and think of it. And then he feels that perhaps
                              there isn't. -- A. A. Milne
                            • Charlie Poole
                              Hi Steven, ... Clearly. I was responding to (what I perceived to be) the level of the team in question. Slicing stories vertically is one of the harder
                              Message 14 of 18 , Jan 2, 2009
                              • 0 Attachment
                                Hi Steven,

                                > > Leaving aside the importance of automated testing - and that's a
                                > > pretty big thing to leave aside - I suggest that you not
                                > schedule any
                                > > stories unless they have "acceptance tests"
                                > >
                                > > In this context, I'm referring to a simple note on the back of the
                                > > story card that says how the story will be verified. If it
                                > won't fit
                                > > there, make it a reference to something else, but be sure that
                                > > something else isn't a full-fledged QA test plan.
                                >
                                > Somewhat better is "if the acceptance test will not fit on
                                > the back of the card, slice the story into substories each of
                                > which can have fit its acceptance test on the back of a card".
                                > In other words, make a story out of each individual acceptance test.

                                Clearly. I was responding to (what I perceived to be) the level
                                of the team in question. Slicing stories "vertically" is one of
                                the harder things for teams to learn.

                                But you're right, and that would be the next thing I'd push
                                toward - probably even before automation of the big tests.

                                Charlie
                              • George Dinwiddie
                                ... Mike, leaving aside the question of regression test automation, it sounds like you may doing the QA testing post iteration. On teams I ve worked with,
                                Message 15 of 18 , Jan 2, 2009
                                • 0 Attachment
                                  Mike Coon wrote:
                                  > In our sprints I have a QA person manually testing bits of functionality as
                                  > they are delivered; This generates defects as we go and we've been adding a
                                  > task to the Sprint backlog for each defect. I'm not completely satisfied
                                  > with this approach, so I am asking what other, more experienced folks are
                                  > doing with defects.

                                  Mike, leaving aside the question of regression test automation, it
                                  sounds like you may doing the QA testing post iteration. On teams I've
                                  worked with, even when the test automation was weak, having the stories
                                  tested within the iteration has proved to be a big help. When a defect
                                  is found, it's just fixed--not added to the backlog. The story isn't
                                  done until the testers (and the PO) are satisfied with it.

                                  - George

                                  --
                                  ----------------------------------------------------------------------
                                  * George Dinwiddie * http://blog.gdinwiddie.com
                                  Software Development http://www.idiacomputing.com
                                  Consultant and Coach http://www.agilemaryland.org
                                  ----------------------------------------------------------------------
                                • Mike Coon
                                  Ron, We are not doing XP though some folks use some practices. We do have a customer; after thinking about my situation here and considering many of the great
                                  Message 16 of 18 , Jan 5, 2009
                                  • 0 Attachment
                                    Ron,

                                    We are not doing XP though some folks use some practices. We do have a
                                    customer; after thinking about my situation here and considering many of the
                                    great responses I plan to focus more on the Product Owner and product
                                    backlog for the next quarter.

                                    As far as ignoring crappy defects - we tried that, and the consultant
                                    working for the dev director punished us with unending meetings and
                                    complaints - now we are bouncing them immediately asking for the steps,
                                    expected and actual results. This still yeilds complaints but immunizes my
                                    folks from much criticism as it's hard to actually say that you want us to
                                    waste time figuring out what you were trying to do and what you thought
                                    would happen.

                                    Lines of power - I am still doing agile adoption as a side gig to my day job
                                    of QA manager. The good news is that i have more influence every day and
                                    can usually get my way. The bad news is that I can usually get my way - so
                                    it helps to be right :-) While I am in my thirtieth year of IT work I'm
                                    still learning just like everybody...

                                    Thanks for the responses.

                                    Mike

                                    On Fri, Jan 2, 2009 at 5:06 PM, Ron Jeffries
                                    <ronjeffries@...>wrote:

                                    > Hello, Mike. On Friday, January 2, 2009, at 1:50:03 PM, you wrote:
                                    >
                                    > > One thing that happens now is we get defects from UAT folks that have not
                                    > > gotten used to the idea that their defect should include steps, expected,
                                    > > and actual results. So we spend a lot of time figuring out what they are
                                    > > reporting which often turns out to be a feature request, training issue,
                                    > or
                                    > > user configuration setting - all of which do need to be identified and
                                    > > resolved but currently take too long to understand, categorize, and
                                    > address.
                                    >
                                    > Are you doing XP? Do you have a customer? What happens if you simply
                                    > ignore alleged defects without info? What are the lines of power on
                                    > the project?
                                    >
                                    > Ron Jeffries
                                    > www.XProgramming.com
                                    > www.xprogramming.com/blog
                                    > Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                                    > of his head. It is, as far as he knows, the only way of coming downstairs,
                                    > but sometimes he feels that there really is another way, if only he could
                                    > stop bumping for a moment and think of it. And then he feels that perhaps
                                    > there isn't. -- A. A. Milne
                                    >
                                    >
                                    >



                                    --
                                    http://mikeonitstuff.net/ New Blog
                                    http://mikeonitstuff.com/ Old Blog
                                    http://mikeonbikes.blogspot.com/


                                    [Non-text portions of this message have been removed]
                                  • Chet Hendrickson
                                    Hello Mike, The first step in an XP style defect resolution process is the creation of concrete acceptance criteria. What Brian Marick calls Examples. These
                                    Message 17 of 18 , Jan 5, 2009
                                    • 0 Attachment
                                      Hello Mike,

                                      The first step in an XP style defect resolution process is the creation of concrete acceptance criteria. What Brian Marick calls Examples. These must belong to the project's Customer, although exactly who creates them is subject to the skills and temperaments of those on the team.
                                      In any case, the developers working on a feature need to have unambiguous guidelines as to how the system will behave once the new feature is added. This works best if they are available early in the development of the feature, but must be in hand before the developers declare the feature done.

                                      Tools like FitNesse are often used for this purpose. We mostly use the one we hate the least at any given moment.

                                      Once the examples exists, any developer who does not make use of them is opening themselves to great ridicule. This goes a long way towards eliminating the 'I thought it was supposed to do this' sort of defect.

                                      This process is not perfect. Often the Customer will come back with 'Yes, that is was I asked for, but this is really what I want'. That case is resolved by the addition of a new sort and its corresponding set of examples.

                                      chet

                                      Monday, January 5, 2009, 8:08:46 AM, you wrote:


                                      Ron,

                                      We are not doing XP though some folks use some practices. We do have a
                                      customer; after thinking about my situation here and considering many of the
                                      great responses I plan to focus more on the Product Owner and product
                                      backlog for the next quarter.

                                      As far as ignoring crappy defects - we tried that, and the consultant
                                      working for the dev director punished us with unending meetings and
                                      complaints - now we are bouncing them immediately asking for the steps,
                                      expected and actual results. This still yeilds complaints but immunizes my
                                      folks from much criticism as it's hard to actually say that you want us to
                                      waste time figuring out what you were trying to do and what you thought
                                      would happen.

                                      Lines of power - I am still doing agile adoption as a side gig to my day job
                                      of QA manager. The good news is that i have more influence every day and
                                      can usually get my way. The bad news is that I can usually get my way - so
                                      it helps to be right :-) While I am in my thirtieth year of IT work I'm
                                      still learning just like everybody...

                                      Thanks for the responses.

                                      Mike

                                      On Fri, Jan 2, 2009 at 5:06 PM, Ron Jeffries
                                      <ronjeffries@...>wrote:

                                      > Hello, Mike. On Friday, January 2, 2009, at 1:50:03 PM, you wrote:
                                      >
                                      > > One thing that happens now is we get defects from UAT folks that have not
                                      > > gotten used to the idea that their defect should include steps, expected,
                                      > > and actual results. So we spend a lot of time figuring out what they are
                                      > > reporting which often turns out to be a feature request, training issue,
                                      > or
                                      > > user configuration setting - all of which do need to be identified and
                                      > > resolved but currently take too long to understand, categorize, and
                                      > address.
                                      >
                                      > Are you doing XP? Do you have a customer? What happens if you simply
                                      > ignore alleged defects without info? What are the lines of power on
                                      > the project?
                                      >
                                      > Ron Jeffries
                                      > www.XProgramming.com
                                      > www.xprogramming.com/blog
                                      > Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                                      > of his head. It is, as far as he knows, the only way of coming downstairs,
                                      > but sometimes he feels that there really is another way, if only he could
                                      > stop bumping for a moment and think of it. And then he feels that perhaps
                                      > there isn't. -- A. A. Milne
                                      >
                                      >
                                      >

                                      --
                                      http://mikeonitstuff.net/ New Blog
                                      http://mikeonitstuff.com/ Old Blog
                                      http://mikeonbikes.blogspot.com/

                                      [Non-text portions of this message have been removed]







                                      --
                                      Best regards,
                                      Chet mailto:lists@...

                                      [Non-text portions of this message have been removed]
                                    • Tim Ottinger
                                      ... /me nods and weeps.
                                      Message 18 of 18 , Jan 6, 2009
                                      • 0 Attachment
                                        > From: Charlie Poole <charlie@...>
                                        > I have to share my own favorite funny/sad excuse for not
                                        > having acceptance tests...
                                        >
                                        > "We don't really know what's wanted, so we can't write a test"
                                        >
                                        > This was stated to me by a programmer who was busily writing
                                        > the code for the story he didn't understand.

                                        /me nods and weeps.
                                      Your message has been successfully submitted and would be delivered to recipients shortly.