Loading ...
Sorry, an error occurred while loading the content.
 

metrics for Scrum roll-out charter

Expand Messages
  • John-Mason P. Shackelford
    The best measures of Scrum success are satisfied customers and a healthy ROI for the business. We d also like some measures which indicate whether a team has
    Message 1 of 7 , Oct 1, 2007
      The best measures of Scrum success are satisfied customers and a
      healthy ROI for the business. We'd also like some measures which
      indicate whether a team has achieved an appropriate level of rhythm
      and predictability in its initial transformation to Scrum. One such
      measure would be the number of consecutive iterations successfully
      completed with all committed stories having been delivered and
      accepted by the product owner. A variation for use on an ongoing basis
      would be the percentage of iterations which complete with all
      committed stories having been accepted by the Product Owner.

      My question is, based on your experience (not speculation please) what
      is a healthy percentage? How many successful iterations (using the
      above definition) before you'd say a team has got it down?

      What other measures do you use to measure the transformation to Scrum
      ? (Another we use is Running Tested Features as shown on the burn up
      signature on a release. The closer we are to linear increase starting
      at the beginning of the cycle rather than a rapid ramp up toward the
      end of the cycle, the less waterfall-ish a team's delivery.)
      --


      John-Mason Shackelford

      Software Development Coach
      Pearson

      2510 North Dodge St.
      Iowa City, IA 52245
      ph. 319-354-9200x6214
      john-mason.shackelford@...
      http://pearsonschool.com
    • Henrik Kniberg
      The idea of finding useful metrics for Scrum is not bad. But the risk is that you might just get what you measure. ... completed ... product owner. This is
      Message 2 of 7 , Oct 1, 2007
        The idea of finding useful metrics for Scrum is not bad. But the risk is that you might just get what you measure.

        > One such measure would be number of consecutive iterations successfully completed
        > with all committed stories having been delivered and accepted by the product owner.

        This is very easy to fulfill - the team just has to under-commit in each sprint. This risks killing the trust that is so vital to Scrum.

        In a healthy Scrum company the teams *usually* deliver exactly what they committed to, but sometimes a bit more and sometimes a bit less. This shows that they really are trying to commit to just the right amount each sprint, and not just tricking the system just to fulfill metrics.

        Would love to be more constructive and suggest alternative metrics, but the gravity field of my bed is exceedingly high right now...

        /Henrik

        --
        Henrik Kniberg
        http://www.crisp.se
        +46 (0)70 492 5284

        On 10/2/07, John-Mason P. Shackelford <jpshack@...> wrote:

        The best measures of Scrum success are satisfied customers and a
        healthy ROI for the business. We'd also like some measures which
        indicate whether a team has achieved an appropriate level of rhythm
        and predictability in its initial transformation to Scrum. One such
        measure would be the number of consecutive iterations successfully
        completed with all committed stories having been delivered and
        accepted by the product owner. A variation for use on an ongoing basis
        would be the percentage of iterations which complete with all
        committed stories having been accepted by the Product Owner.

        My question is, based on your experience (not speculation please) what
        is a healthy percentage? How many successful iterations (using the
        above definition) before you'd say a team has got it down?

        What other measures do you use to measure the transformation to Scrum
        ? (Another we use is Running Tested Features as shown on the burn up
        signature on a release. The closer we are to linear increase starting
        at the beginning of the cycle rather than a rapid ramp up toward the
        end of the cycle, the less waterfall-ish a team's delivery.)
        --

        John-Mason Shackelford

        Software Development Coach
        Pearson

        2510 North Dodge St.
        Iowa City, IA 52245
        ph. 319-354-9200x6214
        john-mason.shackelford@...
        http://pearsonschool.com





      • Henrique Borges
        I sugest the metric Story Delivered / Story Planned . Values near 1 suggest predictability. Higher values sugest under-commit and lower values sugest
        Message 3 of 7 , Oct 1, 2007
          I sugest the metric "Story Delivered / Story Planned".
           
          Values near 1 suggest predictability. Higher values sugest under-commit and lower values sugest over-commitments.

           
          On 10/1/07, Henrik Kniberg <h@...> wrote:

          The idea of finding useful metrics for Scrum is not bad. But the risk is that you might just get what you measure.

          > One such measure would be number of consecutive iterations successfully completed
          > with all committed stories having been delivered and accepted by the product owner.

          This is very easy to fulfill - the team just has to under-commit in each sprint. This risks killing the trust that is so vital to Scrum.

          In a healthy Scrum company the teams *usually* deliver exactly what they committed to, but sometimes a bit more and sometimes a bit less. This shows that they really are trying to commit to just the right amount each sprint, and not just tricking the system just to fulfill metrics.

          Would love to be more constructive and suggest alternative metrics, but the gravity field of my bed is exceedingly high right now...

          /Henrik

          --
          Henrik Kniberg
          http://www.crisp.se
          +46 (0)70 492 5284



          On 10/2/07, John-Mason P. Shackelford <jpshack@... > wrote:

          The best measures of Scrum success are satisfied customers and a
          healthy ROI for the business. We'd also like some measures which
          indicate whether a team has achieved an appropriate level of rhythm
          and predictability in its initial transformation to Scrum. One such
          measure would be the number of consecutive iterations successfully
          completed with all committed stories having been delivered and
          accepted by the product owner. A variation for use on an ongoing basis
          would be the percentage of iterations which complete with all
          committed stories having been accepted by the Product Owner.

          My question is, based on your experience (not speculation please) what
          is a healthy percentage? How many successful iterations (using the
          above definition) before you'd say a team has got it down?

          What other measures do you use to measure the transformation to Scrum
          ? (Another we use is Running Tested Features as shown on the burn up
          signature on a release. The closer we are to linear increase starting
          at the beginning of the cycle rather than a rapid ramp up toward the
          end of the cycle, the less waterfall-ish a team's delivery.)
          --

          John-Mason Shackelford

          Software Development Coach
          Pearson

          2510 North Dodge St.
          Iowa City, IA 52245
          ph. 319-354-9200x6214
          john-mason.shackelford@...
          http://pearsonschool.com








          --
          Henrique Borges
          FAST - Soluções Tecnológicas
        • Daniel Wildt
          You mean the velocity calculation, based on points, hours or stories delivered/planned? That you have with the burndown chart, in Scrum perspective. Regards,
          Message 4 of 7 , Oct 1, 2007
            You mean the velocity calculation, based on points, hours or stories delivered/planned?
             
            That you have with the burndown chart, in Scrum perspective.

            Regards,
            Daniel Wildt
            http://danielwildt.blogspot.com

             


            De: scrumdevelopment@yahoogroups.com [mailto:scrumdevelopment@yahoogroups.com] Em nome de Henrique Borges
            Enviada em: segunda-feira, 1 de outubro de 2007 19:01
            Para: scrumdevelopment@yahoogroups.com
            Assunto: Re: [scrumdevelopment] metrics for Scrum roll-out charter

            I sugest the metric "Story Delivered / Story Planned".
             
            Values near 1 suggest predictability. Higher values sugest under-commit and lower values sugest over-commitments.

             
            On 10/1/07, Henrik Kniberg <h@...> wrote:

            The idea of finding useful metrics for Scrum is not bad. But the risk is that you might just get what you measure.

            > One such measure would be number of consecutive iterations successfully completed
            > with all committed stories having been delivered and accepted by the product owner.

            This is very easy to fulfill - the team just has to under-commit in each sprint. This risks killing the trust that is so vital to Scrum.

            In a healthy Scrum company the teams *usually* deliver exactly what they committed to, but sometimes a bit more and sometimes a bit less. This shows that they really are trying to commit to just the right amount each sprint, and not just tricking the system just to fulfill metrics.

            Would love to be more constructive and suggest alternative metrics, but the gravity field of my bed is exceedingly high right now...

            /Henrik

            --
            Henrik Kniberg
            http://www.crisp. se
            +46 (0)70 492 5284



            On 10/2/07, John-Mason P. Shackelford <jpshack@gmail. com > wrote:

            The best measures of Scrum success are satisfied customers and a
            healthy ROI for the business. We'd also like some measures which
            indicate whether a team has achieved an appropriate level of rhythm
            and predictability in its initial transformation to Scrum. One such
            measure would be the number of consecutive iterations successfully
            completed with all committed stories having been delivered and
            accepted by the product owner. A variation for use on an ongoing basis
            would be the percentage of iterations which complete with all
            committed stories having been accepted by the Product Owner.

            My question is, based on your experience (not speculation please) what
            is a healthy percentage? How many successful iterations (using the
            above definition) before you'd say a team has got it down?

            What other measures do you use to measure the transformation to Scrum
            ? (Another we use is Running Tested Features as shown on the burn up
            signature on a release. The closer we are to linear increase starting
            at the beginning of the cycle rather than a rapid ramp up toward the
            end of the cycle, the less waterfall-ish a team's delivery.)
            --

            John-Mason Shackelford

            Software Development Coach
            Pearson

            2510 North Dodge St.
            Iowa City, IA 52245
            ph. 319-354-9200x6214
            john-mason.shackelf ord@pearson. com
            http://pearsonschoo l.com








            --
            Henrique Borges
            FAST - Soluções Tecnológicas


            No virus found in this incoming message.
            Checked by AVG Free Edition.
            Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date: 30/09/2007 21:01


            No virus found in this outgoing message.
            Checked by AVG Free Edition.
            Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date: 30/09/2007 21:01

          • Malcolm Anderson
            Have you tried a Customer Report Card ? I m not sure where I picked this idea up, but at the time, we were starting to use This new fangled Scrum thing in a
            Message 5 of 7 , Oct 2, 2007
              Have you tried a "Customer Report Card"?

              I'm not sure where I picked this idea up, but at the time, we were
              starting to use "This new fangled Scrum thing" in a maintenance
              organization where the team was handling 4 - 9 different projects in
              any given sprint.

              At some point someone asked, "Well, do our customers even really like
              this new scrum thing?"

              So I suggested a "Customer Report Card" I don't remember the exact
              questions, but I weighted them as heavily as I could in our direction,
              and we gave them to are various different product owners. It looked
              something like this.




              On a Scale of 1 to 10 where 1 is lowest and 10 is highest.

              During the course of the last sprint:

              ____ How well did you feel that the team understood your needs?

              ____ How well did you feel the team performed?

              ____ How well did you feel the team kept you informed?

              ____ How well did you feel the team respond to your requested changes?

              ____ How well did you feel the team explained the impact of any of
              your requested changes?



              I think we determined that anything below an 8 on any measure, was a
              failing grade for the entire report card.



              Now it sounds dorky, but our customers LOVED us for this approch.
              Plus, because we were being measured on it, we made sure to keep the
              Customer Report Card in mind when talking with our product owners.

              Malcolm



              On 10/1/07, John-Mason P. Shackelford <jpshack@...> wrote:
              >
              > The best measures of Scrum success are satisfied customers and a
              > healthy ROI for the business. We'd also like some measures which
              > indicate whether a team has achieved an appropriate level of rhythm
              > and predictability in its initial transformation to Scrum. One such
              > measure would be the number of consecutive iterations successfully
              > completed with all committed stories having been delivered and
              > accepted by the product owner. A variation for use on an ongoing basis
              > would be the percentage of iterations which complete with all
              > committed stories having been accepted by the Product Owner.
              >
              > My question is, based on your experience (not speculation please) what
              > is a healthy percentage? How many successful iterations (using the
              > above definition) before you'd say a team has got it down?
              >
              > What other measures do you use to measure the transformation to Scrum
              > ? (Another we use is Running Tested Features as shown on the burn up
              > signature on a release. The closer we are to linear increase starting
              > at the beginning of the cycle rather than a rapid ramp up toward the
              > end of the cycle, the less waterfall-ish a team's delivery.)
              > --
              >
              > John-Mason Shackelford
              >
              > Software Development Coach
              > Pearson
              >
              > 2510 North Dodge St.
              > Iowa City, IA 52245
              > ph. 319-354-9200x6214
              > john-mason.shackelford@...
              > http://pearsonschool.com
            • Rob Park
              I would expect that stories planned vs. delivered does not account for points, but rather is just a count of story cards. We re working on something similar,
              Message 6 of 7 , Oct 2, 2007
                I would expect that stories planned vs. delivered does not account
                for points, but rather is just a count of story cards.

                We're working on something similar, but at an even grosser level
                of "feature" or "marketing item". Each feature is composed of n
                stories.

                WRT the original question, I have found that when your velocity
                stabilizes is a good time to suggest they're getting it. This will
                fluctuate depending on the team and the iteration length. It was a
                couple years ago for us, so I don't have the exact numbers, but with
                a team of 3 and 1 week iterations we stabilized in a month or less.
                We're now 5, still with 1 week iterations and our last 8 velocities
                were (12, 12, 12, 7, 13, 16, 12, 10). We look at the average of the
                last 8 for our "yesterday's weather", which is even more stable
                (12.0, 12.9, 12.6, 12.0, 11.9, 12.1, 11.8).

                I would also recommend measuring defects / developer-month. It's a
                good guide to pay attention to, but of course may lag other
                indicators such as velocity.

                .rob.


                --- In scrumdevelopment@yahoogroups.com, "Daniel Wildt" <dwildt@...>
                wrote:
                >
                > You mean the velocity calculation, based on points, hours or
                stories delivered/planned?
                >
                > That you have with the burndown chart, in Scrum perspective.
                >
                > Regards,
                > Daniel Wildt
                >
                HYPERLINK "http://danielwildt.blogspot.com"http://danielwildt.blogspot
                .com
                >
                >
                >
                >
                >
                >
                >
                > _____
                >
                > De: scrumdevelopment@yahoogroups.com
                [mailto:scrumdevelopment@yahoogroups.com] Em nome de Henrique Borges
                > Enviada em: segunda-feira, 1 de outubro de 2007 19:01
                > Para: scrumdevelopment@yahoogroups.com
                > Assunto: Re: [scrumdevelopment] metrics for Scrum roll-out charter
                >
                >
                >
                >
                >
                >
                > I sugest the metric "Story Delivered / Story Planned".
                >
                > Values near 1 suggest predictability. Higher values sugest under-
                commit and lower values sugest over-commitments.
                >
                >
                > On 10/1/07, Henrik Kniberg
                <HYPERLINK "mailto:h@..."h@...> wrote:
                >
                > The idea of finding useful metrics for Scrum is not bad. But the
                risk is that you might just get what you measure.
                >
                > > One such measure would be number of consecutive iterations
                successfully completed
                > > with all committed stories having been delivered and accepted by
                the product owner.
                >
                > This is very easy to fulfill - the team just has to under-commit in
                each sprint. This risks killing the trust that is so vital to
                > Scrum.
                >
                > In a healthy Scrum company the teams *usually* deliver exactly what
                they committed to, but sometimes a bit more and sometimes a bit
                > less. This shows that they really are trying to commit to just the
                right amount each sprint, and not just tricking the system just
                > to fulfill metrics.
                >
                > Would love to be more constructive and suggest alternative metrics,
                but the gravity field of my bed is exceedingly high right now...
                >
                > /Henrik
                >
                > --
                > Henrik Kniberg
                > HYPERLINK "http://www.crisp.se/" \nhttp://www.crisp.-se
                > +46 (0)70 492 5284
                >
                >
                >
                >
                > On 10/2/07, John-Mason P. Shackelford
                <HYPERLINK "mailto:jpshack@..." \njpshack@... > wrote:
                >
                > The best measures of Scrum success are satisfied customers and a
                > healthy ROI for the business. We'd also like some measures which
                > indicate whether a team has achieved an appropriate level of rhythm
                > and predictability in its initial transformation to Scrum. One such
                > measure would be the number of consecutive iterations successfully
                > completed with all committed stories having been delivered and
                > accepted by the product owner. A variation for use on an ongoing
                basis
                > would be the percentage of iterations which complete with all
                > committed stories having been accepted by the Product Owner.
                >
                > My question is, based on your experience (not speculation please)
                what
                > is a healthy percentage? How many successful iterations (using the
                > above definition) before you'd say a team has got it down?
                >
                > What other measures do you use to measure the transformation to
                Scrum
                > ? (Another we use is Running Tested Features as shown on the burn up
                > signature on a release. The closer we are to linear increase
                starting
                > at the beginning of the cycle rather than a rapid ramp up toward the
                > end of the cycle, the less waterfall-ish a team's delivery.)
                > --
                >
                > John-Mason Shackelford
                >
                > Software Development Coach
                > Pearson
                >
                > 2510 North Dodge St.
                > Iowa City, IA 52245
                > ph. 319-354-9200x6214
                > HYPERLINK "mailto:john-mason.shackelford%40pearson.com" \njohn-
                mason.shackelf-ord@...
                > HYPERLINK "http://pearsonschool.com/" \nhttp://pearsonschoo-l.com
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                >
                > --
                > Henrique Borges
                > FAST - Soluções Tecnológicas
                >
                >
                >
                >
                >
                >
                > No virus found in this incoming message.
                > Checked by AVG Free Edition.
                > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
                30/09/2007 21:01
                >
                >
                >
                > No virus found in this outgoing message.
                > Checked by AVG Free Edition.
                > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
                30/09/2007 21:01
                >
              • Dan Rawsthorne
                I would like to see how many people worked on each task. Make sure the team is swarming. Make sure that a task is validated by the consumer of the task. I
                Message 7 of 7 , Oct 2, 2007
                  I would like to see how many people worked on each task. Make sure the team is swarming. Make sure that a task is validated by the consumer of the task. I don't know how to collect metrics on this, but make these questions part of the retrospectives. IMHO, most of the metrics you can collect (especially velocity) are trailing indicators of goodness or badness - so we must trust our people to get this sort of info.
                  Dan Rawsthorne, PhD, CST
                  Senior Coach, Danube Technologies
                  dan@..., 425-269-8628


                  Rob Park wrote:

                  I would expect that stories planned vs. delivered does not account
                  for points, but rather is just a count of story cards.

                  We're working on something similar, but at an even grosser level
                  of "feature" or "marketing item". Each feature is composed of n
                  stories.

                  WRT the original question, I have found that when your velocity
                  stabilizes is a good time to suggest they're getting it. This will
                  fluctuate depending on the team and the iteration length. It was a
                  couple years ago for us, so I don't have the exact numbers, but with
                  a team of 3 and 1 week iterations we stabilized in a month or less.
                  We're now 5, still with 1 week iterations and our last 8 velocities
                  were (12, 12, 12, 7, 13, 16, 12, 10). We look at the average of the
                  last 8 for our "yesterday's weather", which is even more stable
                  (12.0, 12.9, 12.6, 12.0, 11.9, 12.1, 11.8).

                  I would also recommend measuring defects / developer-month. It's a
                  good guide to pay attention to, but of course may lag other
                  indicators such as velocity.

                  .rob.

                  --- In scrumdevelopment@ yahoogroups. com, "Daniel Wildt" <dwildt@...>
                  wrote:
                  >
                  > You mean the velocity calculation, based on points, hours or
                  stories delivered/planned?
                  >
                  > That you have with the burndown chart, in Scrum perspective.
                  >
                  > Regards,
                  > Daniel Wildt
                  >
                  HYPERLINK "http://danielwildt. blogspot. com"http://danielwildt. blogspot
                  .com
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  > _____
                  >
                  > De: scrumdevelopment@ yahoogroups. com
                  [mailto:scrumdevelopment@ yahoogroups. com] Em nome de Henrique Borges
                  > Enviada em: segunda-feira, 1 de outubro de 2007 19:01
                  > Para: scrumdevelopment@ yahoogroups. com
                  > Assunto: Re: [scrumdevelopment] metrics for Scrum roll-out charter
                  >
                  >
                  >
                  >
                  >
                  >
                  > I sugest the metric "Story Delivered / Story Planned".
                  >
                  > Values near 1 suggest predictability. Higher values sugest under-
                  commit and lower values sugest over-commitments.
                  >
                  >
                  > On 10/1/07, Henrik Kniberg
                  <HYPERLINK "mailto:h@..."h@...> wrote:
                  >
                  > The idea of finding useful metrics for Scrum is not bad. But the
                  risk is that you might just get what you measure.
                  >
                  > > One such measure would be number of consecutive iterations
                  successfully completed
                  > > with all committed stories having been delivered and accepted by
                  the product owner.
                  >
                  > This is very easy to fulfill - the team just has to under-commit in
                  each sprint. This risks killing the trust that is so vital to
                  > Scrum.
                  >
                  > In a healthy Scrum company the teams *usually* deliver exactly what
                  they committed to, but sometimes a bit more and sometimes a bit
                  > less. This shows that they really are trying to commit to just the
                  right amount each sprint, and not just tricking the system just
                  > to fulfill metrics.
                  >
                  > Would love to be more constructive and suggest alternative metrics,
                  but the gravity field of my bed is exceedingly high right now...
                  >
                  > /Henrik
                  >
                  > --
                  > Henrik Kniberg
                  > HYPERLINK "http://www.crisp. se/" \nhttp://www. crisp.-se
                  > +46 (0)70 492 5284
                  >
                  >
                  >
                  >
                  > On 10/2/07, John-Mason P. Shackelford
                  <HYPERLINK "mailto:jpshack@ ..." \njpshack@.. . > wrote:
                  >
                  > The best measures of Scrum success are satisfied customers and a
                  > healthy ROI for the business. We'd also like some measures which
                  > indicate whether a team has achieved an appropriate level of rhythm
                  > and predictability in its initial transformation to Scrum. One such
                  > measure would be the number of consecutive iterations successfully
                  > completed with all committed stories having been delivered and
                  > accepted by the product owner. A variation for use on an ongoing
                  basis
                  > would be the percentage of iterations which complete with all
                  > committed stories having been accepted by the Product Owner.
                  >
                  > My question is, based on your experience (not speculation please)
                  what
                  > is a healthy percentage? How many successful iterations (using the
                  > above definition) before you'd say a team has got it down?
                  >
                  > What other measures do you use to measure the transformation to
                  Scrum
                  > ? (Another we use is Running Tested Features as shown on the burn up
                  > signature on a release. The closer we are to linear increase
                  starting
                  > at the beginning of the cycle rather than a rapid ramp up toward the
                  > end of the cycle, the less waterfall-ish a team's delivery.)
                  > --
                  >
                  > John-Mason Shackelford
                  >
                  > Software Development Coach
                  > Pearson
                  >
                  > 2510 North Dodge St.
                  > Iowa City, IA 52245
                  > ph. 319-354-9200x6214
                  > HYPERLINK "mailto:john- mason.shackelfor d%40pearson. com" \njohn-
                  mason.shackelf- ord@...
                  > HYPERLINK "http://pearsonschoo l.com/" \nhttp://pearsonscho o-l.com
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  > --
                  > Henrique Borges
                  > FAST - Soluções Tecnológicas
                  >
                  >
                  >
                  >
                  >
                  >
                  > No virus found in this incoming message.
                  > Checked by AVG Free Edition.
                  > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
                  30/09/2007 21:01
                  >
                  >
                  >
                  > No virus found in this outgoing message.
                  > Checked by AVG Free Edition.
                  > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
                  30/09/2007 21:01
                  >

                Your message has been successfully submitted and would be delivered to recipients shortly.