Loading ...
Sorry, an error occurred while loading the content.
 

Re: Story - QA Score

Expand Messages
  • JackM
    Hi Brian, Some good questions .... here s my 2 cents. Your team needs to setup your Done criteria for each story. This includes acceptance tests and quality
    Message 1 of 14 , Dec 23, 2010
      Hi Brian,

      Some good questions .... here's my 2 cents.

      Your team needs to setup your "Done" criteria for each story. This includes acceptance tests and quality criteria. If you still have bugs remaining for a story at the end of the sprint then the story is not done. You have to complete the development and test and fix cycle all within the same sprint. This will put you in good shape quality wise.

      If you're having problems in this area, you either have not defined your acceptance test criteria and your done criteria properly, or qa is not involved from the getgo i.e. they should already be part of the planning meeting right up front. Or you could be taking on too many stories so you're not getting enough time to get your stuff done.

      Please do not reward individuals. This is a dangerous route to take. Scrum/XP are team "sports". Team should be rewarded.

      Hope this helps
      Jack
      www.agilebuddy.com
      blog.agilebuddy.com

      --- In scrumdevelopment@yahoogroups.com, "Brian" <bplawlor34@...> wrote:
      >
      > Our Dev Team is having trouble completing Stories with low bug counts at the end of our Iteration. An Executive suggested, even if we fix the problem, creating a Story - QA Score.
      >
      > We can then utilize this Score to determine how successful the Dev Team is each Iteration, and with each Story. We can also use the Score to monitor the performance of the individual programmers, and maybe reward those who complete Stories with High Grades.
      >
      > Is this a good idea? I'm sure it'll make some programmers nervous, but I doubt my Ace Programmers will be concerned at all. If anything, they'll enjoy the rewards.
      >
      > I would also love some ideas for making this effective, assuming if this is a good idea.
      >
      > Thanks in advance for the feedback.
      > -Brian
      >
    • Kiran
      ... if no bugs come back. Of course the whole question makes me think you may be building things in one Sprint and then testing them in the next. That s not
      Message 2 of 14 , Dec 24, 2010
        >>>>I do like the idea of counting stories as done only
        if no bugs come
        back. Of course the whole question makes me think you may be
        building things in one Sprint and then testing them in the next.
        That's not the way: things are supposed to be done and ready to ship
        in a single Sprint.

        I am not sure as how this happens in large projects.However I have seen most of the time this do not happen in large project or programs, where multiple teams from different vendors work for various reasons.

        Squeezing all the activities like development, deployment.testing and QA in 3 weeks sprint is hard job.This finally hits the quality.

        As regard to OP, I would say something is amiss in his process.either stories have been wrongly estimated or something else.and yes rewarding individual just because of low bug count is just not right.if someone is lagging back him up and bring him to speed.

        Just my thoughts and wish you all a merry Christmas and prosperous new year.

        On 12/23/2010 1:21 PM, Ron Jeffries wrote:
         

        Hello, Brian. On Thursday, December 23, 2010, at 12:10:28 PM, you
        wrote:

        > Our Dev Team is having trouble completing Stories with low bug
        > counts at the end of our Iteration. An Executive suggested, even
        > if we fix the problem, creating a Story - QA Score.
        >
        I do like the idea of counting stories as done only if no bugs come
        back. Of course the whole question makes me think you may be
        building things in one Sprint and then testing them in the next.
        That's not the way: things are supposed to be done and ready to ship
        in a single Sprint.

        > We can then utilize this Score to determine how successful the
        > Dev Team is each Iteration, and with each Story. We can also use
        > the Score to monitor the performance of the individual
        > programmers, and maybe reward those who complete Stories with High Grades.

        No, no, a thousand times no. If you compensate people this way they
        will gang bang the system. Read "Punished by Rewards" for lots on
        this.

        > Is this a good idea? I'm sure it'll make some programmers
        > nervous, but I doubt my Ace Programmers will be concerned at all.
        > If anything, they'll enjoy the rewards.

        Scrum is a team sport, not an Ace Programmers sport. This notion is
        quite concerning. Also, as others have suggested, Ace Programmers
        may not be as Ace as you might think. My mind thought this:

        "Ace Programmers? What's that? How long is the longest class these
        Ace Programmers have written? What's the average? What is their
        unit test coverage? How frequently are QA finding bugs?"

        Ron Jeffries
        www.XProgramming.com
        I'm giving the best advice I have. You get to decide whether it's true for you.



        -- 
        With Regards,
        Kiran Badi
        Email:kiran@...
        Ph-India-(+91)9823378726
        Ph-UK-(+44)2070239006
        Ph- US-(+01)6178307605
        
      • peterskeide
        Like others have said, science is not really in favour of rewarding knowledge workers beyond what is considered fair for the work they do. People in our line
        Message 3 of 14 , Dec 25, 2010
          Like others have said, science is not really in favour of rewarding knowledge workers beyond what is considered fair for the work they do. People in our line of business are often motivated by the work itself. You can avoid limiting this intrinsic motivation by giving workers control aspects of their work such as when to work, where to work, how to do the work and also who to work with (important when assembling a team).

          After reading your description of the problem, I ask myself the following questions:

          Have you targeted the quality issues in team retrospectives? If so, and it's still not improving, how can you change the way you run the retrospectives to generate more/different insight and make the team understand that they own the problem?

          Are your team really acting as a team? If some "ace" programmers consistently get stories to Done and without bugs, how can they help others raise the quality of their work?

          Are some of the teammembers weak on domain-knowledge? Things you did not know that you did not know tends to resurface later as bugs.

          I see a lot of people mentioning testing and acceptance criteria. Tests are fine, but not enough. Are you focusing on defect prevention in addition to detection? Try some (slightly) formal code inspections in combination with checklists. Checklists are a greay way of sharing knowledge. Code inspections done right are another great way of sharing knowledge. Make sure everyone has their code inspected from time to time (the "aces" as well). Results of the inspections are for the team only. Managers need/should not know.

          How often do the teammembers pair program? Pairing is effective for knowledge sharing (but not really a substitute for code inspections - use both).

          Are you using static/dynamic code analysis tools (such as Checkstyle for Java)? If not, suggest that your team start using them. Most such tools contain loads of useful rules for code quality, and can often be customized to comply with team coding standards. Junior programmers can learn a lot from such tools.

          A nice sideeffect of code analysis tools I have seen firsthand is how they can overcome "social" barriers where some teammembers do not respond well to comments about their code from other teammembers. It appears to be easier to take feedback from a purely objective tool.

          --- In scrumdevelopment@yahoogroups.com, "Brian" <bplawlor34@...> wrote:
          >
          > Our Dev Team is having trouble completing Stories with low bug counts at the end of our Iteration. An Executive suggested, even if we fix the problem, creating a Story - QA Score.
          >
          > We can then utilize this Score to determine how successful the Dev Team is each Iteration, and with each Story. We can also use the Score to monitor the performance of the individual programmers, and maybe reward those who complete Stories with High Grades.
          >
          > Is this a good idea? I'm sure it'll make some programmers nervous, but I doubt my Ace Programmers will be concerned at all. If anything, they'll enjoy the rewards.
          >
          > I would also love some ideas for making this effective, assuming if this is a good idea.
          >
          > Thanks in advance for the feedback.
          > -Brian
          >
        • George Dinwiddie
          Kiran, ... Yes, that s hard. It s easier with a shorter sprint. Really. You do, of course, have to approach the work a little differently, but the shorter
          Message 4 of 14 , Dec 25, 2010
            Kiran,

            On 12/25/10 2:09 AM, Kiran wrote:
            > Squeezing all the activities like development, deployment.testing and QA
            > in 3 weeks sprint is hard job.This finally hits the quality.

            Yes, that's hard. It's easier with a shorter sprint. Really. You do,
            of course, have to approach the work a little differently, but the
            shorter sprint helps people do that.

            - George

            --
            ----------------------------------------------------------------------
            * George Dinwiddie * http://blog.gdinwiddie.com
            Software Development http://www.idiacomputing.com
            Consultant and Coach http://www.agilemaryland.org
            ----------------------------------------------------------------------
          • George Dinwiddie
            Brian, ... I ve seen the responses to this (with which I agree), but I can t help but wonder what the Executive is thinking, here. How would such a score be
            Message 5 of 14 , Dec 27, 2010
              Brian,

              On 12/23/10 12:10 PM, Brian wrote:
              > Our Dev Team is having trouble completing Stories with low bug counts
              > at the end of our Iteration. An Executive suggested, even if we fix
              > the problem, creating a Story - QA Score.
              >
              > We can then utilize this Score to determine how successful the Dev
              > Team is each Iteration, and with each Story. We can also use the
              > Score to monitor the performance of the individual programmers, and
              > maybe reward those who complete Stories with High Grades.

              I've seen the responses to this (with which I agree), but I can't help
              but wonder what the Executive is thinking, here. How would such a score
              be calculated? This seems to presume that people are working as
              individuals rather than as a team. (That, alone, could be a major
              contributor to the problem of either not completing stories or releasing
              them with bugs.) I'm certainly missing much about the context.

              Counting bugs discovered after the end of the sprint is a good measure
              to track, in my opinion. I would graph that over time, and look at the
              shape of the curve. I would also do root-cause-analysis (5 whys) as
              each is discovered to figure out how the /process/ can be improved to
              prevent similar occurrences in the future. (This is much more valuable
              than assigning blame.)

              - George

              --
              ----------------------------------------------------------------------
              * George Dinwiddie * http://blog.gdinwiddie.com
              Software Development http://www.idiacomputing.com
              Consultant and Coach http://www.agilemaryland.org
              ----------------------------------------------------------------------
            • Brian
              I guess I should elaborate a little bit more, however, I think you all have expressed the concern I was looking for to utilize as ammo back up the ladder. We
              Message 6 of 14 , Dec 27, 2010
                I guess I should elaborate a little bit more, however, I think you all have expressed the concern I was looking for to utilize as ammo back up the ladder.

                We have 4-week Iterations; which includes QA. We do Unit and Buddy testing, as well as all the regular QA, integration, regression testing. We do not have automated scripts yet, but will have in a month.

                It does sound like the Exec's will push for some sort of "Story Quality Score". I'm not sure I can stop that. But, I may be able to sway them towards focusing on the Story itself... avoiding individual grading.

                We'll see...

                Thanks to all of you for taking the time to lend your advice. Our Company has by migrating towards being more Agile after working in a Waterfall environment for 30 years. I guess we have some growing pains to work through still.

                -Brian
              • scrumnoob
                Hi Brian As has already been asked/stated, I to would be interested to know how this conversation/issue played out at the retrospective(s). It is within the
                Message 7 of 14 , Dec 29, 2010
                  Hi Brian

                  As has already been asked/stated, I to would be interested to know how this conversation/issue played out at the retrospective(s).

                  It is within the gift of the team to work out how to resolve whatever quality issue there maybe, I would say it is their responsibility also.

                  How has the issue manifest itself to execs if your defintion of done/done includes all the levels of QA you mention?

                  Best of luck

                  Sean


                  --- In scrumdevelopment@yahoogroups.com, "Brian" <bplawlor34@...> wrote:
                  >
                  > I guess I should elaborate a little bit more, however, I think you all have expressed the concern I was looking for to utilize as ammo back up the ladder.
                  >
                  > We have 4-week Iterations; which includes QA. We do Unit and Buddy testing, as well as all the regular QA, integration, regression testing. We do not have automated scripts yet, but will have in a month.
                  >
                  > It does sound like the Exec's will push for some sort of "Story Quality Score". I'm not sure I can stop that. But, I may be able to sway them towards focusing on the Story itself... avoiding individual grading.
                  >
                  > We'll see...
                  >
                  > Thanks to all of you for taking the time to lend your advice. Our Company has by migrating towards being more Agile after working in a Waterfall environment for 30 years. I guess we have some growing pains to work through still.
                  >
                  > -Brian
                  >
                • Brian
                  I think there s a combination of multiple incorrect practices which have been going on...to be honest. - Developers tackling too many Stories - Focus on
                  Message 8 of 14 , Dec 31, 2010
                    I think there's a combination of multiple incorrect practices which have been going on...to be honest.
                    - Developers tackling too many Stories
                    - Focus on Functionality, not incorporating Visuals/Treatment
                    - Dev Team not Project focused
                    - Dev Team lacking required WPF coding experience

                    The Retrospectives are active discussions, but they are repeating. Everyone recognizes the problems, but I guess we're still trying to improve our Iterative practices. Many of the discussions, which involved high bug counts which need to be eliminated, repeated. However, consequences for not succeeding in the goals were never implemented, and that area is outside my control.

                    Take multiple iterations of trying to scale back with quantity of Stories, but seeing no improvements, plus a frustrated Exec Branch, and you get discussions for a Story Quality Score to grade Dev performance.

                    However, I think I have successfully talked them into a different path. We're going to take 1 Iteration and get caught up with all Bugs and lingering issues. The following Iteration, I'm creating a very transparent Bug Burn Down Monitoring Chart.

                    I've also worked hand-in-hand, walking the Dev Team through the Stories. I'm taking it a step further to facilitate proper Story production by only handing over the specific Stories they will start with in the Iteration. Once one is noted as Done (a shipable product), I'll provide the next Story. This should eliminate working on too many at one time. That will happen for the Iteration after next, when they're back to Stories.

                    The expertise of the Dev Team has improved and we have a couple more on the way. To be honest, I think we may be in good shape. Proof is in the pudding of course, but I believe I was able to keep the Exec's at bay one last time. If Dev Team doesn't follow what I've outline, they will get the butts kicked next time.


                    Thanks for everyone's feedback BTW.
                    -Brian

                    --- In scrumdevelopment@yahoogroups.com, "scrumnoob" <scrumnoob@...> wrote:
                    >
                    > Hi Brian
                    >
                    > As has already been asked/stated, I to would be interested to know how this conversation/issue played out at the retrospective(s).
                    >
                    > It is within the gift of the team to work out how to resolve whatever quality issue there maybe, I would say it is their responsibility also.
                    >
                    > How has the issue manifest itself to execs if your defintion of done/done includes all the levels of QA you mention?
                    >
                    > Best of luck
                    >
                    > Sean
                    >
                    >
                    > --- In scrumdevelopment@yahoogroups.com, "Brian" <bplawlor34@> wrote:
                    > >
                    > > I guess I should elaborate a little bit more, however, I think you all have expressed the concern I was looking for to utilize as ammo back up the ladder.
                    > >
                    > > We have 4-week Iterations; which includes QA. We do Unit and Buddy testing, as well as all the regular QA, integration, regression testing. We do not have automated scripts yet, but will have in a month.
                    > >
                    > > It does sound like the Exec's will push for some sort of "Story Quality Score". I'm not sure I can stop that. But, I may be able to sway them towards focusing on the Story itself... avoiding individual grading.
                    > >
                    > > We'll see...
                    > >
                    > > Thanks to all of you for taking the time to lend your advice. Our Company has by migrating towards being more Agile after working in a Waterfall environment for 30 years. I guess we have some growing pains to work through still.
                    > >
                    > > -Brian
                    > >
                    >
                  • peterskeide
                    It may be that I m overreacting to your choice of words, but I get a bit worried when you write things like consequences for not succeeding in the
                    Message 9 of 14 , Jan 3, 2011
                      It may be that I'm overreacting to your choice of words, but I get a bit worried when you write things like "consequences for not succeeding in the (retrospective) goals" and "they will get the butts kicked next time".

                      If you are changing your development process from a phased/waterfall type to agile and at the same time introducing a new technology, a lot of people have to relearn how to do parts of their work. To ease the "pain" of such a transition, safety in the workplace is very important. Failure must be acceptable; a source of information to be used for process improvement.

                      I get the impression that some of your management do not think this way. Consider very carefully if you have sufficient management support at the right level for the agile initiative. Also, try to gauge the level of managements understanding of what makes scrum/agile work. If managers are still thinking about individual developer performance benchmarks, it is a clear sign of the need for coaching at a different level than development.

                      --- In scrumdevelopment@yahoogroups.com, "Brian" <bplawlor34@...> wrote:
                      >
                      > I think there's a combination of multiple incorrect practices which have been going on...to be honest.
                      > - Developers tackling too many Stories
                      > - Focus on Functionality, not incorporating Visuals/Treatment
                      > - Dev Team not Project focused
                      > - Dev Team lacking required WPF coding experience
                      >
                      > The Retrospectives are active discussions, but they are repeating. Everyone recognizes the problems, but I guess we're still trying to improve our Iterative practices. Many of the discussions, which involved high bug counts which need to be eliminated, repeated. However, consequences for not succeeding in the goals were never implemented, and that area is outside my control.
                      >
                      > Take multiple iterations of trying to scale back with quantity of Stories, but seeing no improvements, plus a frustrated Exec Branch, and you get discussions for a Story Quality Score to grade Dev performance.
                      >
                      > However, I think I have successfully talked them into a different path. We're going to take 1 Iteration and get caught up with all Bugs and lingering issues. The following Iteration, I'm creating a very transparent Bug Burn Down Monitoring Chart.
                      >
                      > I've also worked hand-in-hand, walking the Dev Team through the Stories. I'm taking it a step further to facilitate proper Story production by only handing over the specific Stories they will start with in the Iteration. Once one is noted as Done (a shipable product), I'll provide the next Story. This should eliminate working on too many at one time. That will happen for the Iteration after next, when they're back to Stories.
                      >
                      > The expertise of the Dev Team has improved and we have a couple more on the way. To be honest, I think we may be in good shape. Proof is in the pudding of course, but I believe I was able to keep the Exec's at bay one last time. If Dev Team doesn't follow what I've outline, they will get the butts kicked next time.
                      >
                      >
                      > Thanks for everyone's feedback BTW.
                      > -Brian
                      >
                      > --- In scrumdevelopment@yahoogroups.com, "scrumnoob" <scrumnoob@> wrote:
                      > >
                      > > Hi Brian
                      > >
                      > > As has already been asked/stated, I to would be interested to know how this conversation/issue played out at the retrospective(s).
                      > >
                      > > It is within the gift of the team to work out how to resolve whatever quality issue there maybe, I would say it is their responsibility also.
                      > >
                      > > How has the issue manifest itself to execs if your defintion of done/done includes all the levels of QA you mention?
                      > >
                      > > Best of luck
                      > >
                      > > Sean
                      > >
                      > >
                      > > --- In scrumdevelopment@yahoogroups.com, "Brian" <bplawlor34@> wrote:
                      > > >
                      > > > I guess I should elaborate a little bit more, however, I think you all have expressed the concern I was looking for to utilize as ammo back up the ladder.
                      > > >
                      > > > We have 4-week Iterations; which includes QA. We do Unit and Buddy testing, as well as all the regular QA, integration, regression testing. We do not have automated scripts yet, but will have in a month.
                      > > >
                      > > > It does sound like the Exec's will push for some sort of "Story Quality Score". I'm not sure I can stop that. But, I may be able to sway them towards focusing on the Story itself... avoiding individual grading.
                      > > >
                      > > > We'll see...
                      > > >
                      > > > Thanks to all of you for taking the time to lend your advice. Our Company has by migrating towards being more Agile after working in a Waterfall environment for 30 years. I guess we have some growing pains to work through still.
                      > > >
                      > > > -Brian
                      > > >
                      > >
                      >
                    Your message has been successfully submitted and would be delivered to recipients shortly.