Loading ...
Sorry, an error occurred while loading the content.

Re: [scrumdevelopment] metrics for Scrum roll-out charter

Expand Messages
  • Malcolm Anderson
    Have you tried a Customer Report Card ? I m not sure where I picked this idea up, but at the time, we were starting to use This new fangled Scrum thing in a
    Message 1 of 7 , Oct 2, 2007
    • 0 Attachment
      Have you tried a "Customer Report Card"?

      I'm not sure where I picked this idea up, but at the time, we were
      starting to use "This new fangled Scrum thing" in a maintenance
      organization where the team was handling 4 - 9 different projects in
      any given sprint.

      At some point someone asked, "Well, do our customers even really like
      this new scrum thing?"

      So I suggested a "Customer Report Card" I don't remember the exact
      questions, but I weighted them as heavily as I could in our direction,
      and we gave them to are various different product owners. It looked
      something like this.




      On a Scale of 1 to 10 where 1 is lowest and 10 is highest.

      During the course of the last sprint:

      ____ How well did you feel that the team understood your needs?

      ____ How well did you feel the team performed?

      ____ How well did you feel the team kept you informed?

      ____ How well did you feel the team respond to your requested changes?

      ____ How well did you feel the team explained the impact of any of
      your requested changes?



      I think we determined that anything below an 8 on any measure, was a
      failing grade for the entire report card.



      Now it sounds dorky, but our customers LOVED us for this approch.
      Plus, because we were being measured on it, we made sure to keep the
      Customer Report Card in mind when talking with our product owners.

      Malcolm



      On 10/1/07, John-Mason P. Shackelford <jpshack@...> wrote:
      >
      > The best measures of Scrum success are satisfied customers and a
      > healthy ROI for the business. We'd also like some measures which
      > indicate whether a team has achieved an appropriate level of rhythm
      > and predictability in its initial transformation to Scrum. One such
      > measure would be the number of consecutive iterations successfully
      > completed with all committed stories having been delivered and
      > accepted by the product owner. A variation for use on an ongoing basis
      > would be the percentage of iterations which complete with all
      > committed stories having been accepted by the Product Owner.
      >
      > My question is, based on your experience (not speculation please) what
      > is a healthy percentage? How many successful iterations (using the
      > above definition) before you'd say a team has got it down?
      >
      > What other measures do you use to measure the transformation to Scrum
      > ? (Another we use is Running Tested Features as shown on the burn up
      > signature on a release. The closer we are to linear increase starting
      > at the beginning of the cycle rather than a rapid ramp up toward the
      > end of the cycle, the less waterfall-ish a team's delivery.)
      > --
      >
      > John-Mason Shackelford
      >
      > Software Development Coach
      > Pearson
      >
      > 2510 North Dodge St.
      > Iowa City, IA 52245
      > ph. 319-354-9200x6214
      > john-mason.shackelford@...
      > http://pearsonschool.com
    • Rob Park
      I would expect that stories planned vs. delivered does not account for points, but rather is just a count of story cards. We re working on something similar,
      Message 2 of 7 , Oct 2, 2007
      • 0 Attachment
        I would expect that stories planned vs. delivered does not account
        for points, but rather is just a count of story cards.

        We're working on something similar, but at an even grosser level
        of "feature" or "marketing item". Each feature is composed of n
        stories.

        WRT the original question, I have found that when your velocity
        stabilizes is a good time to suggest they're getting it. This will
        fluctuate depending on the team and the iteration length. It was a
        couple years ago for us, so I don't have the exact numbers, but with
        a team of 3 and 1 week iterations we stabilized in a month or less.
        We're now 5, still with 1 week iterations and our last 8 velocities
        were (12, 12, 12, 7, 13, 16, 12, 10). We look at the average of the
        last 8 for our "yesterday's weather", which is even more stable
        (12.0, 12.9, 12.6, 12.0, 11.9, 12.1, 11.8).

        I would also recommend measuring defects / developer-month. It's a
        good guide to pay attention to, but of course may lag other
        indicators such as velocity.

        .rob.


        --- In scrumdevelopment@yahoogroups.com, "Daniel Wildt" <dwildt@...>
        wrote:
        >
        > You mean the velocity calculation, based on points, hours or
        stories delivered/planned?
        >
        > That you have with the burndown chart, in Scrum perspective.
        >
        > Regards,
        > Daniel Wildt
        >
        HYPERLINK "http://danielwildt.blogspot.com"http://danielwildt.blogspot
        .com
        >
        >
        >
        >
        >
        >
        >
        > _____
        >
        > De: scrumdevelopment@yahoogroups.com
        [mailto:scrumdevelopment@yahoogroups.com] Em nome de Henrique Borges
        > Enviada em: segunda-feira, 1 de outubro de 2007 19:01
        > Para: scrumdevelopment@yahoogroups.com
        > Assunto: Re: [scrumdevelopment] metrics for Scrum roll-out charter
        >
        >
        >
        >
        >
        >
        > I sugest the metric "Story Delivered / Story Planned".
        >
        > Values near 1 suggest predictability. Higher values sugest under-
        commit and lower values sugest over-commitments.
        >
        >
        > On 10/1/07, Henrik Kniberg
        <HYPERLINK "mailto:h@..."h@...> wrote:
        >
        > The idea of finding useful metrics for Scrum is not bad. But the
        risk is that you might just get what you measure.
        >
        > > One such measure would be number of consecutive iterations
        successfully completed
        > > with all committed stories having been delivered and accepted by
        the product owner.
        >
        > This is very easy to fulfill - the team just has to under-commit in
        each sprint. This risks killing the trust that is so vital to
        > Scrum.
        >
        > In a healthy Scrum company the teams *usually* deliver exactly what
        they committed to, but sometimes a bit more and sometimes a bit
        > less. This shows that they really are trying to commit to just the
        right amount each sprint, and not just tricking the system just
        > to fulfill metrics.
        >
        > Would love to be more constructive and suggest alternative metrics,
        but the gravity field of my bed is exceedingly high right now...
        >
        > /Henrik
        >
        > --
        > Henrik Kniberg
        > HYPERLINK "http://www.crisp.se/" \nhttp://www.crisp.-se
        > +46 (0)70 492 5284
        >
        >
        >
        >
        > On 10/2/07, John-Mason P. Shackelford
        <HYPERLINK "mailto:jpshack@..." \njpshack@... > wrote:
        >
        > The best measures of Scrum success are satisfied customers and a
        > healthy ROI for the business. We'd also like some measures which
        > indicate whether a team has achieved an appropriate level of rhythm
        > and predictability in its initial transformation to Scrum. One such
        > measure would be the number of consecutive iterations successfully
        > completed with all committed stories having been delivered and
        > accepted by the product owner. A variation for use on an ongoing
        basis
        > would be the percentage of iterations which complete with all
        > committed stories having been accepted by the Product Owner.
        >
        > My question is, based on your experience (not speculation please)
        what
        > is a healthy percentage? How many successful iterations (using the
        > above definition) before you'd say a team has got it down?
        >
        > What other measures do you use to measure the transformation to
        Scrum
        > ? (Another we use is Running Tested Features as shown on the burn up
        > signature on a release. The closer we are to linear increase
        starting
        > at the beginning of the cycle rather than a rapid ramp up toward the
        > end of the cycle, the less waterfall-ish a team's delivery.)
        > --
        >
        > John-Mason Shackelford
        >
        > Software Development Coach
        > Pearson
        >
        > 2510 North Dodge St.
        > Iowa City, IA 52245
        > ph. 319-354-9200x6214
        > HYPERLINK "mailto:john-mason.shackelford%40pearson.com" \njohn-
        mason.shackelf-ord@...
        > HYPERLINK "http://pearsonschool.com/" \nhttp://pearsonschoo-l.com
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        > --
        > Henrique Borges
        > FAST - Soluções Tecnológicas
        >
        >
        >
        >
        >
        >
        > No virus found in this incoming message.
        > Checked by AVG Free Edition.
        > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
        30/09/2007 21:01
        >
        >
        >
        > No virus found in this outgoing message.
        > Checked by AVG Free Edition.
        > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
        30/09/2007 21:01
        >
      • Dan Rawsthorne
        I would like to see how many people worked on each task. Make sure the team is swarming. Make sure that a task is validated by the consumer of the task. I
        Message 3 of 7 , Oct 2, 2007
        • 0 Attachment
          I would like to see how many people worked on each task. Make sure the team is swarming. Make sure that a task is validated by the consumer of the task. I don't know how to collect metrics on this, but make these questions part of the retrospectives. IMHO, most of the metrics you can collect (especially velocity) are trailing indicators of goodness or badness - so we must trust our people to get this sort of info.
          Dan Rawsthorne, PhD, CST
          Senior Coach, Danube Technologies
          dan@..., 425-269-8628


          Rob Park wrote:

          I would expect that stories planned vs. delivered does not account
          for points, but rather is just a count of story cards.

          We're working on something similar, but at an even grosser level
          of "feature" or "marketing item". Each feature is composed of n
          stories.

          WRT the original question, I have found that when your velocity
          stabilizes is a good time to suggest they're getting it. This will
          fluctuate depending on the team and the iteration length. It was a
          couple years ago for us, so I don't have the exact numbers, but with
          a team of 3 and 1 week iterations we stabilized in a month or less.
          We're now 5, still with 1 week iterations and our last 8 velocities
          were (12, 12, 12, 7, 13, 16, 12, 10). We look at the average of the
          last 8 for our "yesterday's weather", which is even more stable
          (12.0, 12.9, 12.6, 12.0, 11.9, 12.1, 11.8).

          I would also recommend measuring defects / developer-month. It's a
          good guide to pay attention to, but of course may lag other
          indicators such as velocity.

          .rob.

          --- In scrumdevelopment@ yahoogroups. com, "Daniel Wildt" <dwildt@...>
          wrote:
          >
          > You mean the velocity calculation, based on points, hours or
          stories delivered/planned?
          >
          > That you have with the burndown chart, in Scrum perspective.
          >
          > Regards,
          > Daniel Wildt
          >
          HYPERLINK "http://danielwildt. blogspot. com"http://danielwildt. blogspot
          .com
          >
          >
          >
          >
          >
          >
          >
          > _____
          >
          > De: scrumdevelopment@ yahoogroups. com
          [mailto:scrumdevelopment@ yahoogroups. com] Em nome de Henrique Borges
          > Enviada em: segunda-feira, 1 de outubro de 2007 19:01
          > Para: scrumdevelopment@ yahoogroups. com
          > Assunto: Re: [scrumdevelopment] metrics for Scrum roll-out charter
          >
          >
          >
          >
          >
          >
          > I sugest the metric "Story Delivered / Story Planned".
          >
          > Values near 1 suggest predictability. Higher values sugest under-
          commit and lower values sugest over-commitments.
          >
          >
          > On 10/1/07, Henrik Kniberg
          <HYPERLINK "mailto:h@..."h@...> wrote:
          >
          > The idea of finding useful metrics for Scrum is not bad. But the
          risk is that you might just get what you measure.
          >
          > > One such measure would be number of consecutive iterations
          successfully completed
          > > with all committed stories having been delivered and accepted by
          the product owner.
          >
          > This is very easy to fulfill - the team just has to under-commit in
          each sprint. This risks killing the trust that is so vital to
          > Scrum.
          >
          > In a healthy Scrum company the teams *usually* deliver exactly what
          they committed to, but sometimes a bit more and sometimes a bit
          > less. This shows that they really are trying to commit to just the
          right amount each sprint, and not just tricking the system just
          > to fulfill metrics.
          >
          > Would love to be more constructive and suggest alternative metrics,
          but the gravity field of my bed is exceedingly high right now...
          >
          > /Henrik
          >
          > --
          > Henrik Kniberg
          > HYPERLINK "http://www.crisp. se/" \nhttp://www. crisp.-se
          > +46 (0)70 492 5284
          >
          >
          >
          >
          > On 10/2/07, John-Mason P. Shackelford
          <HYPERLINK "mailto:jpshack@ ..." \njpshack@.. . > wrote:
          >
          > The best measures of Scrum success are satisfied customers and a
          > healthy ROI for the business. We'd also like some measures which
          > indicate whether a team has achieved an appropriate level of rhythm
          > and predictability in its initial transformation to Scrum. One such
          > measure would be the number of consecutive iterations successfully
          > completed with all committed stories having been delivered and
          > accepted by the product owner. A variation for use on an ongoing
          basis
          > would be the percentage of iterations which complete with all
          > committed stories having been accepted by the Product Owner.
          >
          > My question is, based on your experience (not speculation please)
          what
          > is a healthy percentage? How many successful iterations (using the
          > above definition) before you'd say a team has got it down?
          >
          > What other measures do you use to measure the transformation to
          Scrum
          > ? (Another we use is Running Tested Features as shown on the burn up
          > signature on a release. The closer we are to linear increase
          starting
          > at the beginning of the cycle rather than a rapid ramp up toward the
          > end of the cycle, the less waterfall-ish a team's delivery.)
          > --
          >
          > John-Mason Shackelford
          >
          > Software Development Coach
          > Pearson
          >
          > 2510 North Dodge St.
          > Iowa City, IA 52245
          > ph. 319-354-9200x6214
          > HYPERLINK "mailto:john- mason.shackelfor d%40pearson. com" \njohn-
          mason.shackelf- ord@...
          > HYPERLINK "http://pearsonschoo l.com/" \nhttp://pearsonscho o-l.com
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          >
          > --
          > Henrique Borges
          > FAST - Soluções Tecnológicas
          >
          >
          >
          >
          >
          >
          > No virus found in this incoming message.
          > Checked by AVG Free Edition.
          > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
          30/09/2007 21:01
          >
          >
          >
          > No virus found in this outgoing message.
          > Checked by AVG Free Edition.
          > Version: 7.5.488 / Virus Database: 269.13.35/1040 - Release Date:
          30/09/2007 21:01
          >

        Your message has been successfully submitted and would be delivered to recipients shortly.