## Art of estimating stories:

Expand Messages
• Hi, We are in the middle of an iteration, and I just realized again we underestimated stories. The problem is that velocity does not really helps, because 40%
Message 1 of 7 , Jan 26, 2001
Hi,

We are in the middle of an iteration, and I just realized again we
underestimated
stories. The problem is that velocity does not really helps, because

40% of all stories are overestimated by more than twice
30% of all stories are underestimated by more than twice

Our team velocity is really wild from iteration to iteration
It is like this:

1.1
2.1
3.25
1.4

The planning book (Planning Extreme Programming) suggegsts we reestimate
the stories.
Fine, we can do it, but which values would go to the final calculation
of velocity, the original
or the reestimated? And if we end up reestimating majority of our user
stories,
how can the "customer" trust our estimates?

Are we doing something wrong?
Do you have any suggestions?

Thank you!

Michael.
• ... [...] ... [...] ... Unfortunately, they won t be able to trust your estimates, but that s okay, because your estimates have been wrong and they shouldn t
Message 2 of 7 , Jan 26, 2001
On 2001.01.26, Michael Larionov <michael.larionov@...> wrote:
> We are in the middle of an iteration, and I just realized again we
> underestimated stories.
[...]
> 40% of all stories are overestimated by more than twice
> 30% of all stories are underestimated by more than twice
>
> Our team velocity is really wild from iteration to iteration
[...]
> And if we end up reestimating majority of our user
> stories, how can the "customer" trust our estimates?

Unfortunately, they won't be able to trust your estimates, but
that's okay, because your estimates have been wrong and they
shouldn't trust your estimates. What would be worse, though,
more stories done in an iteration than your team can actually
handle? What's worse -- not being 100% confident in estimates,
or expecting more stories done in an iteration than can actually
be done? This is up to the customer, but be open and communicate
and in the end let the customer decide, that way you empower the
customer to make decisions which will be more valuable to you
in the long run. (Even if they can't trust the estimates, they'll
be able to trust you and your team -- that confidence can go
a long way ... they might even try and help you work on making
more accurate estimates if they have that confidence in you.)

> Are we doing something wrong? Do you have any suggestions?

I wouldn't say you're doing something "wrong" since I really
have no idea what you're doing, but I'll take a whack at a
guess:

Break them up into tasks, and estimate the tasks -- use the sum
of the tasks as the stand-in estimate for the story. Any task
that a developer isn't "pretty confident" in their estimate should
be examined to try and figure out what makes them not confident
in their estimate - unfamiliar with the technology, or the problem
domain, or whatever. Try quick 10-30 minute spike solutions for
tasks that are hard to estimate: 30 minutes to find out that a
task can be done in a day is more valuable time spent than
overestimating 3 days for that task. Same thing applies in the
reverse direction, where a developer thinks a task will be much
easier than it actually turns out to be -- the spike will give
them a much better idea of how tough it actually is.

Try estimating tasks in pairs (not necessarily the pair that
will work on the task) -- if one person says "oh, 4 days"
and the person he/she is paired with says "I'm pretty sure
it can be done in 2 ..." they can discuss why each thinks
it can be done in the amount of time they think it'll take,
and maybe come to a consensus as to how long it might really
take (one person might not be seeing a quicker solution that
the other person is thinking of, etc.) ... in the end, the
pair could spike a quick solution, which might pay off as
well.

Hope this helps ... if you give more details as to how your
team goes through the estimation process and some thoughts
your team has when estimating, I might be able to give some
more meaningful feedback ...

- Dossy

--
Dossy Shiobara mail: dossy@...
Panoptic Computer Network web: http://www.panoptic.com/
• On Fri, 26 Jan 2001 18:21:58 -0500, Michael Larionov ... While introducing XP, we faced similar symptomes due to two factors: - our tasks were too large, which
Message 3 of 7 , Jan 27, 2001
On Fri, 26 Jan 2001 18:21:58 -0500, Michael Larionov
<michael.larionov@...> wrote:

>We are in the middle of an iteration, and I just realized again we
>underestimated
>stories. The problem is that velocity does not really helps, because
>
>40% of all stories are overestimated by more than twice
>30% of all stories are underestimated by more than twice
>
>Our team velocity is really wild from iteration to iteration
>It is like this:
>
>1.1
>2.1
>3.25
>1.4
>
>The planning book (Planning Extreme Programming) suggegsts we reestimate
>the stories.
>Fine, we can do it, but which values would go to the final calculation
>of velocity, the original
>or the reestimated? And if we end up reestimating majority of our user
>stories,
>how can the "customer" trust our estimates?
>
>Are we doing something wrong?
>Do you have any suggestions?

While introducing XP, we faced similar symptomes due to two factors:

> one should break down stories into tasks that developers feel
really comfortable with, especially when one is
trying out a (new) combination of new technologies.
Breaking down stories to a finer granularity yields
more raw material for statistics, which tends to
smoothe the curves.

- our iterations were too short (2 weeks), in combination
with way too large tasks this sometimes prevented some developers
from completing even a single task within an iteration

> one can try out longer iterations (increment them by
a week or so), which also tends to smoothe the curves

Hope this helps, best regards,
Rolf
• Thank you for you post! I have a few remaining questions: 1) If the story estimate is the sum of the tasks estimates, then we have to break the stories into
Message 4 of 7 , Jan 29, 2001
Thank you for you post!

I have a few remaining questions:

1) If the story estimate is the sum of the tasks estimates,
then we have to break the stories into tasks when we do
release planning rather than iteration planning.
This will be unnecessary overhead because the "customer"
may not choose all the stories for a release. In fact, he may
intentionally include much more stories just to have time
estimates from the team, which will make it easier for him
to choose the stories for the next release.

2) You were right that the overestimated stories happens to
be the ones which involve much of the technical risk.
In fact, most of the overestimated stories are related to PL/SQL programming,
and most of the underestimated stories are related to JSP.
Those are underestimated becuase of huge feature creep for such
projects. Is it acceptable to keep two velocities, one for PL/SQL stories, the other
for
JSP stories?

3) Still the question remanes: when we reestimate the story,
is it the old estimate or the new one which contributes to the project velocity?

Thank you!

Michael.

Dossy wrote:

> On 2001.01.26, Michael Larionov <michael.larionov@...> wrote:
> > We are in the middle of an iteration, and I just realized again we
> > underestimated stories.
> [...]
> > 40% of all stories are overestimated by more than twice
> > 30% of all stories are underestimated by more than twice
> >
> > Our team velocity is really wild from iteration to iteration
> [...]
> > And if we end up reestimating majority of our user
> > stories, how can the "customer" trust our estimates?
>
> Unfortunately, they won't be able to trust your estimates, but
> that's okay, because your estimates have been wrong and they
> shouldn't trust your estimates. What would be worse, though,
> more stories done in an iteration than your team can actually
> handle? What's worse -- not being 100% confident in estimates,
> or expecting more stories done in an iteration than can actually
> be done? This is up to the customer, but be open and communicate
> and in the end let the customer decide, that way you empower the
> customer to make decisions which will be more valuable to you
> in the long run. (Even if they can't trust the estimates, they'll
> be able to trust you and your team -- that confidence can go
> a long way ... they might even try and help you work on making
> more accurate estimates if they have that confidence in you.)
>
> > Are we doing something wrong? Do you have any suggestions?
>
> I wouldn't say you're doing something "wrong" since I really
> have no idea what you're doing, but I'll take a whack at a
> guess:
>
> Break them up into tasks, and estimate the tasks -- use the sum
> of the tasks as the stand-in estimate for the story. Any task
> that a developer isn't "pretty confident" in their estimate should
> be examined to try and figure out what makes them not confident
> in their estimate - unfamiliar with the technology, or the problem
> domain, or whatever. Try quick 10-30 minute spike solutions for
> tasks that are hard to estimate: 30 minutes to find out that a
> task can be done in a day is more valuable time spent than
> overestimating 3 days for that task. Same thing applies in the
> reverse direction, where a developer thinks a task will be much
> easier than it actually turns out to be -- the spike will give
> them a much better idea of how tough it actually is.
>
> Try estimating tasks in pairs (not necessarily the pair that
> will work on the task) -- if one person says "oh, 4 days"
> and the person he/she is paired with says "I'm pretty sure
> it can be done in 2 ..." they can discuss why each thinks
> it can be done in the amount of time they think it'll take,
> and maybe come to a consensus as to how long it might really
> take (one person might not be seeing a quicker solution that
> the other person is thinking of, etc.) ... in the end, the
> pair could spike a quick solution, which might pay off as
> well.
>
> Hope this helps ... if you give more details as to how your
> team goes through the estimation process and some thoughts
> your team has when estimating, I might be able to give some
> more meaningful feedback ...
>
> - Dossy
>
> --
> Dossy Shiobara mail: dossy@...
> Panoptic Computer Network web: http://www.panoptic.com/
>
> To Post a message, send it to: extremeprogramming@...
>
> To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
>

[Non-text portions of this message have been removed]
• ... Yes, though I would not recommend a formal breakdown, just an informal one during discussion ... ... It s necessary to do whatever it takes to get the
Message 5 of 7 , Jan 29, 2001
At 02:15 PM 1/29/2001 -0500, it seemed like Michael Larionov wrote:
>1) If the story estimate is the sum of the tasks estimates,
>then we have to break the stories into tasks when we do
>release planning rather than iteration planning.

Yes, though I would not recommend a formal breakdown, just an informal one
during discussion ...

>This will be unnecessary overhead because the "customer"
>may not choose all the stories for a release. In fact, he may
>intentionally include much more stories just to have time
>estimates from the team, which will make it easier for him
>to choose the stories for the next release.

It's necessary to do whatever it takes to get the estimates linear, i.e. to
make a 2 take twice as long as a 1.

>2) You were right that the overestimated stories happens to
>be the ones which involve much of the technical risk.
>In fact, most of the overestimated stories are related to PL/SQL programming,
>and most of the underestimated stories are related to JSP.
>Those are underestimated becuase of huge feature creep for such
>projects. Is it acceptable to keep two velocities, one for PL/SQL
>stories, the other
>for
>JSP stories?

Sure, but if you keep feeding back your performance into your estimates, I
would expect the estimates to get more accurate without keeping separate
velocities.

And if a story is high risk, consider working on it early and consider
doing a spike.

Regards,

Ronald E Jeffries
http://www.XProgramming.com
http://www.objectmentor.com
• ... You re welcome! I hope at least some of what I said made sense and was helpful! ... A customer who prioritizes stories based on the amount of time it
Message 6 of 7 , Jan 29, 2001
On 2001.01.29, Michael Larionov <michael.larionov@...> wrote:
> Thank you for you post!

You're welcome! I hope at least some of what I said made sense and

> I have a few remaining questions:
>
> 1) [...] [Breaking story estimates into task estimates] will be
> unnecessary overhead because the "customer" may not choose all the
> stories for a release. In fact, he may intentionally include much more
> stories just to have time estimates from the team, which will make it
> easier for him to choose the stories for the next release.

A customer who prioritizes stories based on the amount of time it
takes to implement (how soon it can be released) as opposed to the
absolute amount of business value that story delivers is foolish,
in my opinion*. (* read on...)

If story 1 is more valuable than story 2, even if story 2 would be
quicker to deliver ... the customer should still make story 1 top
priority ... doing it the other way around is "paying more for less"
which is silly. If you look at all time spent as "paying" and
request a less valuable story over a more valuable story just because
you can get more of the less valuable stories done in the same amount
of time because they're faster/the estimates are lower ... then
you're paying the same price for something less valuable.

Of course, if you clump stories together, and say that "this
collection of very fast-to-complete stories would have more
business value than this other set of fewer, but slower to
complete stories" then maybe my suggested rule of thumb above
doesn't hold true. At some point, you have to learn where the
line is drawn ...

> 2) You were right that the overestimated stories happens to
> be the ones which involve much of the technical risk.
[...]
> Is it acceptable to keep two velocities, one for PL/SQL stories, the
> other for JSP stories?

Read on ... I don't think it's necessary (don't really know about
"acceptable" -- try and see how it works for you) and I explain
why in my next paragraph ...

> 3) Still the question remanes: when we reestimate the story, is it the
> old estimate or the new one which contributes to the project velocity?

I'm not sure ... I don't bother re-estimating stories once the
iteration planning is done. Even wild bumps in your velocity
due to incredibly poor estimates get smoothed out over time.
Unless it's a matter of life and death, I don't let poor estimates
get me down -- I let the customer know as soon as I know that the
estimate was poor, and that we might need to drop a story from
the current iteration ... then try my hardest to get as close to
the iteration plan as possible. And, try not to make the same
mistake in the next and future iterations ...

- Dossy

--
Dossy Shiobara mail: dossy@...
Panoptic Computer Network web: http://www.panoptic.com/
• ... I want to disagree on this. If the customer has fixed cost he always have to do a trade off between the story value and story cost. Maximizing ROI, if you
Message 7 of 7 , Jan 30, 2001
Dossy wrote:

> A customer who prioritizes stories based on the amount of time it
> takes to implement (how soon it can be released) as opposed to the
> absolute amount of business value that story delivers is foolish,
> in my opinion*. (* read on...)
>
> If story 1 is more valuable than story 2, even if story 2 would be
> quicker to deliver ... the customer should still make story 1 top
> priority ... doing it the other way around is "paying more for less"
> which is silly. If you look at all time spent as "paying" and
> request a less valuable story over a more valuable story just because
> you can get more of the less valuable stories done in the same amount
> of time because they're faster/the estimates are lower ... then
> you're paying the same price for something less valuable.
>

I want to disagree on this. If the customer has fixed cost
he always have to do a trade off between the story value and story cost.
Maximizing ROI, if you want. So it is quite possible the customer
will substitute a costly story for a less valuable but cheaper one,
so ROI will be higher.

Thank you!

Michael.

[Non-text portions of this message have been removed]
Your message has been successfully submitted and would be delivered to recipients shortly.