Re: [scrumdevelopment] Re: newbie Q about testing time estimates
- It's not so much that estimating testing is hard, (the included
message and other follow ups describe the estimation) but that
estimates for test are not precis ( as opposed to accurate). The
problem is that while estimated/actual effort developing test cases is
somewhat normally distrubuted, the resolution of problems is neither
normal nor symetric. Some have used a log normal distribution to model
fix time, though I believe the tail follows a Pareto. In either case
most fix times are less than the mean (the distrubution is highly
assymetric after all). In the cased of a Pareto tail, total fix time
will be dominated by a small number of hard to resolve problems. It's
not unusual for 10% of the problems to consume 50% of the fix time.
The 80-20 principle (loosely Pareto's rule) is a fair starting point.
To those used to dealing with symetric distributions, the log-normal
and Pareto character of defect fix times may seem chaotic. In fact, we
can still obtain a mean fix time, but the variance is often
(mathematiclly) infinite. A weaker form of the Central Limit Theorem
applies so that aggregated fix times will still converge to the mean,
albeit more slowly than for better behaved distributions.
Fix times have a high variance that will be significantly
underestimated if we exclude the outliers from the analysis. These
outliers make estimation seem harder than it is.
On Tue, 21 Dec 2004 21:47:31 -0000, Lisa Crispin <lisa.crispin@...> wrote:
> Estimating is hard for everyone; I don't know why it should be any
> harder for testing than for coding. Mike's approach of specifying
> testing tasks on a card, eg. "write FitNesse tests for the XYZ
> story" really helps. If this isn't enough to help you estimate,
> you can break it down even further: "Work with product owner to
> define details of story", "Define test data", "Specify test cases".
> The problem with that is that you may end up spending too much time
> to come up with the estimate. But it might be a way to learn to