Loading ...
Sorry, an error occurred while loading the content.
 

Art of estimating stories:

Expand Messages
  • Michael Larionov
    Hi, We are in the middle of an iteration, and I just realized again we underestimated stories. The problem is that velocity does not really helps, because 40%
    Message 1 of 7 , Jan 26, 2001
      Hi,

      We are in the middle of an iteration, and I just realized again we
      underestimated
      stories. The problem is that velocity does not really helps, because

      40% of all stories are overestimated by more than twice
      30% of all stories are underestimated by more than twice

      Our team velocity is really wild from iteration to iteration
      It is like this:

      1.1
      2.1
      3.25
      1.4

      The planning book (Planning Extreme Programming) suggegsts we reestimate
      the stories.
      Fine, we can do it, but which values would go to the final calculation
      of velocity, the original
      or the reestimated? And if we end up reestimating majority of our user
      stories,
      how can the "customer" trust our estimates?

      Are we doing something wrong?
      Do you have any suggestions?

      Thank you!

      Michael.
    • Dossy
      ... [...] ... [...] ... Unfortunately, they won t be able to trust your estimates, but that s okay, because your estimates have been wrong and they shouldn t
      Message 2 of 7 , Jan 26, 2001
        On 2001.01.26, Michael Larionov <michael.larionov@...> wrote:
        > We are in the middle of an iteration, and I just realized again we
        > underestimated stories.
        [...]
        > 40% of all stories are overestimated by more than twice
        > 30% of all stories are underestimated by more than twice
        >
        > Our team velocity is really wild from iteration to iteration
        [...]
        > And if we end up reestimating majority of our user
        > stories, how can the "customer" trust our estimates?

        Unfortunately, they won't be able to trust your estimates, but
        that's okay, because your estimates have been wrong and they
        shouldn't trust your estimates. What would be worse, though,
        that they mistakenly trust your estimates and instead expect
        more stories done in an iteration than your team can actually
        handle? What's worse -- not being 100% confident in estimates,
        or expecting more stories done in an iteration than can actually
        be done? This is up to the customer, but be open and communicate
        and in the end let the customer decide, that way you empower the
        customer to make decisions which will be more valuable to you
        in the long run. (Even if they can't trust the estimates, they'll
        be able to trust you and your team -- that confidence can go
        a long way ... they might even try and help you work on making
        more accurate estimates if they have that confidence in you.)

        > Are we doing something wrong? Do you have any suggestions?

        I wouldn't say you're doing something "wrong" since I really
        have no idea what you're doing, but I'll take a whack at a
        guess:

        Your stories are too complicated for your developers to estimate.
        Break them up into tasks, and estimate the tasks -- use the sum
        of the tasks as the stand-in estimate for the story. Any task
        that a developer isn't "pretty confident" in their estimate should
        be examined to try and figure out what makes them not confident
        in their estimate - unfamiliar with the technology, or the problem
        domain, or whatever. Try quick 10-30 minute spike solutions for
        tasks that are hard to estimate: 30 minutes to find out that a
        task can be done in a day is more valuable time spent than
        overestimating 3 days for that task. Same thing applies in the
        reverse direction, where a developer thinks a task will be much
        easier than it actually turns out to be -- the spike will give
        them a much better idea of how tough it actually is.

        Try estimating tasks in pairs (not necessarily the pair that
        will work on the task) -- if one person says "oh, 4 days"
        and the person he/she is paired with says "I'm pretty sure
        it can be done in 2 ..." they can discuss why each thinks
        it can be done in the amount of time they think it'll take,
        and maybe come to a consensus as to how long it might really
        take (one person might not be seeing a quicker solution that
        the other person is thinking of, etc.) ... in the end, the
        pair could spike a quick solution, which might pay off as
        well.


        Hope this helps ... if you give more details as to how your
        team goes through the estimation process and some thoughts
        your team has when estimating, I might be able to give some
        more meaningful feedback ...

        - Dossy

        --
        Dossy Shiobara mail: dossy@...
        Panoptic Computer Network web: http://www.panoptic.com/
      • Rolf F. Katzenberger
        On Fri, 26 Jan 2001 18:21:58 -0500, Michael Larionov ... While introducing XP, we faced similar symptomes due to two factors: - our tasks were too large, which
        Message 3 of 7 , Jan 27, 2001
          On Fri, 26 Jan 2001 18:21:58 -0500, Michael Larionov
          <michael.larionov@...> wrote:

          >We are in the middle of an iteration, and I just realized again we
          >underestimated
          >stories. The problem is that velocity does not really helps, because
          >
          >40% of all stories are overestimated by more than twice
          >30% of all stories are underestimated by more than twice
          >
          >Our team velocity is really wild from iteration to iteration
          >It is like this:
          >
          >1.1
          >2.1
          >3.25
          >1.4
          >
          >The planning book (Planning Extreme Programming) suggegsts we reestimate
          >the stories.
          >Fine, we can do it, but which values would go to the final calculation
          >of velocity, the original
          >or the reestimated? And if we end up reestimating majority of our user
          >stories,
          >how can the "customer" trust our estimates?
          >
          >Are we doing something wrong?
          >Do you have any suggestions?

          While introducing XP, we faced similar symptomes due to two factors:

          - our tasks were too large, which made estimates unreliable

          > one should break down stories into tasks that developers feel
          really comfortable with, especially when one is
          trying out a (new) combination of new technologies.
          Breaking down stories to a finer granularity yields
          more raw material for statistics, which tends to
          smoothe the curves.

          - our iterations were too short (2 weeks), in combination
          with way too large tasks this sometimes prevented some developers
          from completing even a single task within an iteration

          > one can try out longer iterations (increment them by
          a week or so), which also tends to smoothe the curves


          Hope this helps, best regards,
          Rolf
        • Michael Larionov
          Thank you for you post! I have a few remaining questions: 1) If the story estimate is the sum of the tasks estimates, then we have to break the stories into
          Message 4 of 7 , Jan 29, 2001
            Thank you for you post!

            I have a few remaining questions:

            1) If the story estimate is the sum of the tasks estimates,
            then we have to break the stories into tasks when we do
            release planning rather than iteration planning.
            This will be unnecessary overhead because the "customer"
            may not choose all the stories for a release. In fact, he may
            intentionally include much more stories just to have time
            estimates from the team, which will make it easier for him
            to choose the stories for the next release.

            2) You were right that the overestimated stories happens to
            be the ones which involve much of the technical risk.
            In fact, most of the overestimated stories are related to PL/SQL programming,
            and most of the underestimated stories are related to JSP.
            Those are underestimated becuase of huge feature creep for such
            projects. Is it acceptable to keep two velocities, one for PL/SQL stories, the other
            for
            JSP stories?

            3) Still the question remanes: when we reestimate the story,
            is it the old estimate or the new one which contributes to the project velocity?

            Thank you!

            Michael.

            Dossy wrote:

            > On 2001.01.26, Michael Larionov <michael.larionov@...> wrote:
            > > We are in the middle of an iteration, and I just realized again we
            > > underestimated stories.
            > [...]
            > > 40% of all stories are overestimated by more than twice
            > > 30% of all stories are underestimated by more than twice
            > >
            > > Our team velocity is really wild from iteration to iteration
            > [...]
            > > And if we end up reestimating majority of our user
            > > stories, how can the "customer" trust our estimates?
            >
            > Unfortunately, they won't be able to trust your estimates, but
            > that's okay, because your estimates have been wrong and they
            > shouldn't trust your estimates. What would be worse, though,
            > that they mistakenly trust your estimates and instead expect
            > more stories done in an iteration than your team can actually
            > handle? What's worse -- not being 100% confident in estimates,
            > or expecting more stories done in an iteration than can actually
            > be done? This is up to the customer, but be open and communicate
            > and in the end let the customer decide, that way you empower the
            > customer to make decisions which will be more valuable to you
            > in the long run. (Even if they can't trust the estimates, they'll
            > be able to trust you and your team -- that confidence can go
            > a long way ... they might even try and help you work on making
            > more accurate estimates if they have that confidence in you.)
            >
            > > Are we doing something wrong? Do you have any suggestions?
            >
            > I wouldn't say you're doing something "wrong" since I really
            > have no idea what you're doing, but I'll take a whack at a
            > guess:
            >
            > Your stories are too complicated for your developers to estimate.
            > Break them up into tasks, and estimate the tasks -- use the sum
            > of the tasks as the stand-in estimate for the story. Any task
            > that a developer isn't "pretty confident" in their estimate should
            > be examined to try and figure out what makes them not confident
            > in their estimate - unfamiliar with the technology, or the problem
            > domain, or whatever. Try quick 10-30 minute spike solutions for
            > tasks that are hard to estimate: 30 minutes to find out that a
            > task can be done in a day is more valuable time spent than
            > overestimating 3 days for that task. Same thing applies in the
            > reverse direction, where a developer thinks a task will be much
            > easier than it actually turns out to be -- the spike will give
            > them a much better idea of how tough it actually is.
            >
            > Try estimating tasks in pairs (not necessarily the pair that
            > will work on the task) -- if one person says "oh, 4 days"
            > and the person he/she is paired with says "I'm pretty sure
            > it can be done in 2 ..." they can discuss why each thinks
            > it can be done in the amount of time they think it'll take,
            > and maybe come to a consensus as to how long it might really
            > take (one person might not be seeing a quicker solution that
            > the other person is thinking of, etc.) ... in the end, the
            > pair could spike a quick solution, which might pay off as
            > well.
            >
            > Hope this helps ... if you give more details as to how your
            > team goes through the estimation process and some thoughts
            > your team has when estimating, I might be able to give some
            > more meaningful feedback ...
            >
            > - Dossy
            >
            > --
            > Dossy Shiobara mail: dossy@...
            > Panoptic Computer Network web: http://www.panoptic.com/
            >
            > To Post a message, send it to: extremeprogramming@...
            >
            > To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
            >
            > Ad-free courtesy of objectmentor.com


            [Non-text portions of this message have been removed]
          • Ron Jeffries
            ... Yes, though I would not recommend a formal breakdown, just an informal one during discussion ... ... It s necessary to do whatever it takes to get the
            Message 5 of 7 , Jan 29, 2001
              At 02:15 PM 1/29/2001 -0500, it seemed like Michael Larionov wrote:
              >1) If the story estimate is the sum of the tasks estimates,
              >then we have to break the stories into tasks when we do
              >release planning rather than iteration planning.

              Yes, though I would not recommend a formal breakdown, just an informal one
              during discussion ...

              >This will be unnecessary overhead because the "customer"
              >may not choose all the stories for a release. In fact, he may
              >intentionally include much more stories just to have time
              >estimates from the team, which will make it easier for him
              >to choose the stories for the next release.

              It's necessary to do whatever it takes to get the estimates linear, i.e. to
              make a 2 take twice as long as a 1.

              >2) You were right that the overestimated stories happens to
              >be the ones which involve much of the technical risk.
              >In fact, most of the overestimated stories are related to PL/SQL programming,
              >and most of the underestimated stories are related to JSP.
              >Those are underestimated becuase of huge feature creep for such
              >projects. Is it acceptable to keep two velocities, one for PL/SQL
              >stories, the other
              >for
              >JSP stories?

              Sure, but if you keep feeding back your performance into your estimates, I
              would expect the estimates to get more accurate without keeping separate
              velocities.

              And if a story is high risk, consider working on it early and consider
              doing a spike.

              Regards,



              Ronald E Jeffries
              http://www.XProgramming.com
              http://www.objectmentor.com
            • Dossy
              ... You re welcome! I hope at least some of what I said made sense and was helpful! ... A customer who prioritizes stories based on the amount of time it
              Message 6 of 7 , Jan 29, 2001
                On 2001.01.29, Michael Larionov <michael.larionov@...> wrote:
                > Thank you for you post!

                You're welcome! I hope at least some of what I said made sense and
                was helpful!

                > I have a few remaining questions:
                >
                > 1) [...] [Breaking story estimates into task estimates] will be
                > unnecessary overhead because the "customer" may not choose all the
                > stories for a release. In fact, he may intentionally include much more
                > stories just to have time estimates from the team, which will make it
                > easier for him to choose the stories for the next release.

                A customer who prioritizes stories based on the amount of time it
                takes to implement (how soon it can be released) as opposed to the
                absolute amount of business value that story delivers is foolish,
                in my opinion*. (* read on...)

                If story 1 is more valuable than story 2, even if story 2 would be
                quicker to deliver ... the customer should still make story 1 top
                priority ... doing it the other way around is "paying more for less"
                which is silly. If you look at all time spent as "paying" and
                request a less valuable story over a more valuable story just because
                you can get more of the less valuable stories done in the same amount
                of time because they're faster/the estimates are lower ... then
                you're paying the same price for something less valuable.

                Of course, if you clump stories together, and say that "this
                collection of very fast-to-complete stories would have more
                business value than this other set of fewer, but slower to
                complete stories" then maybe my suggested rule of thumb above
                doesn't hold true. At some point, you have to learn where the
                line is drawn ...


                > 2) You were right that the overestimated stories happens to
                > be the ones which involve much of the technical risk.
                [...]
                > Is it acceptable to keep two velocities, one for PL/SQL stories, the
                > other for JSP stories?

                Read on ... I don't think it's necessary (don't really know about
                "acceptable" -- try and see how it works for you) and I explain
                why in my next paragraph ...


                > 3) Still the question remanes: when we reestimate the story, is it the
                > old estimate or the new one which contributes to the project velocity?

                I'm not sure ... I don't bother re-estimating stories once the
                iteration planning is done. Even wild bumps in your velocity
                due to incredibly poor estimates get smoothed out over time.
                Unless it's a matter of life and death, I don't let poor estimates
                get me down -- I let the customer know as soon as I know that the
                estimate was poor, and that we might need to drop a story from
                the current iteration ... then try my hardest to get as close to
                the iteration plan as possible. And, try not to make the same
                mistake in the next and future iterations ...


                - Dossy

                --
                Dossy Shiobara mail: dossy@...
                Panoptic Computer Network web: http://www.panoptic.com/
              • Michael Larionov
                ... I want to disagree on this. If the customer has fixed cost he always have to do a trade off between the story value and story cost. Maximizing ROI, if you
                Message 7 of 7 , Jan 30, 2001
                  Dossy wrote:

                  > A customer who prioritizes stories based on the amount of time it
                  > takes to implement (how soon it can be released) as opposed to the
                  > absolute amount of business value that story delivers is foolish,
                  > in my opinion*. (* read on...)
                  >
                  > If story 1 is more valuable than story 2, even if story 2 would be
                  > quicker to deliver ... the customer should still make story 1 top
                  > priority ... doing it the other way around is "paying more for less"
                  > which is silly. If you look at all time spent as "paying" and
                  > request a less valuable story over a more valuable story just because
                  > you can get more of the less valuable stories done in the same amount
                  > of time because they're faster/the estimates are lower ... then
                  > you're paying the same price for something less valuable.
                  >

                  I want to disagree on this. If the customer has fixed cost
                  he always have to do a trade off between the story value and story cost.
                  Maximizing ROI, if you want. So it is quite possible the customer
                  will substitute a costly story for a less valuable but cheaper one,
                  so ROI will be higher.

                  Thank you!

                  Michael.


                  [Non-text portions of this message have been removed]
                Your message has been successfully submitted and would be delivered to recipients shortly.