Loading ...
Sorry, an error occurred while loading the content.

Re: [scrumdevelopment] Re: newbie Q about testing time estimates

Expand Messages
  • William Nichols
    It s not so much that estimating testing is hard, (the included message and other follow ups describe the estimation) but that estimates for test are not
    Message 1 of 10 , Dec 24, 2004
    • 0 Attachment
      It's not so much that estimating testing is hard, (the included
      message and other follow ups describe the estimation) but that
      estimates for test are not precis ( as opposed to accurate). The
      problem is that while estimated/actual effort developing test cases is
      somewhat normally distrubuted, the resolution of problems is neither
      normal nor symetric. Some have used a log normal distribution to model
      fix time, though I believe the tail follows a Pareto. In either case
      most fix times are less than the mean (the distrubution is highly
      assymetric after all). In the cased of a Pareto tail, total fix time
      will be dominated by a small number of hard to resolve problems. It's
      not unusual for 10% of the problems to consume 50% of the fix time.
      The 80-20 principle (loosely Pareto's rule) is a fair starting point.

      To those used to dealing with symetric distributions, the log-normal
      and Pareto character of defect fix times may seem chaotic. In fact, we
      can still obtain a mean fix time, but the variance is often
      (mathematiclly) infinite. A weaker form of the Central Limit Theorem
      applies so that aggregated fix times will still converge to the mean,
      albeit more slowly than for better behaved distributions.

      Fix times have a high variance that will be significantly
      underestimated if we exclude the outliers from the analysis. These
      outliers make estimation seem harder than it is.

      Bill Nichols



      On Tue, 21 Dec 2004 21:47:31 -0000, Lisa Crispin <lisa.crispin@...> wrote:
      >
      > Estimating is hard for everyone; I don't know why it should be any
      > harder for testing than for coding. Mike's approach of specifying
      > testing tasks on a card, eg. "write FitNesse tests for the XYZ
      > story" really helps. If this isn't enough to help you estimate,
      > you can break it down even further: "Work with product owner to
      > define details of story", "Define test data", "Specify test cases".
      > The problem with that is that you may end up spending too much time
      > to come up with the estimate. But it might be a way to learn to
      > estimate.
      >
      ...
    Your message has been successfully submitted and would be delivered to recipients shortly.