Loading ...
Sorry, an error occurred while loading the content.
 

Re: [XP] The relationship between coding effort and fixing effort

Expand Messages
  • Manuel Klimek
    John, I see. Good point. I think I d try to integrate this into the increase of coding effort factor. My idea was to do a worst case analysis. This is why
    Message 1 of 6 , Aug 28, 2007
      John,

      I see. Good point. I think I'd try to integrate this into the "increase of
      coding effort"
      factor. My idea was to do a "worst case" analysis. This is why I think my
      crude
      oversimplification is not too bad.

      I can easily convince people that a factor of two for the up-front coding
      effort is
      the worst case for both TDD and PP. For PP it's obvious, and my own TDD
      experience makes me believe in what I read a lot: you write about twice the
      code, so twice the effort seems to be a reasonable upper limit. Everybody
      I know agrees that the up-front effort is /not more/ than a factor of 2.

      I believe finding a tenth of the errors that currently slip unstructured
      testing to be quite worst case, too. I don't know for sure, but my gut tells
      me I find a lot more. Since it should be "worst case", I just tried with a
      tenth to see what happens.

      An defect rate of 20 / kLOC is hopefully a worst case assumption in the
      sense
      that I don't expect unstructured testing to be a lot better. Since the
      opportunity
      cost of letting defects go wild is smaller, if the defect rate is smaller, I
      tried with
      20, again, just to see what happens.

      To get back to your response: Of course I ignored some of the cost savings
      that TDD/PP obviously bring. Reading IEEE Software gives me the feeling that
      the additional initial effort is probably a lot lower than factor 2, because
      of
      the effects you mentioned, amongst others.

      The interesting thing (for me) is that even in this "worst case" scenario I
      get what I'd consider a relatively low limit for introducing practices that
      lower the defect rate: I think there are a lot of environments where the
      cost of an error in the field is a lot higher than 500 times the cost of a
      line of code without using the practice.

      As a side node, I find that TDD is not replacing /all/ of my debugging
      effort /if/ I code a complicated algorithm. I'm doing a test driven Sudoku
      (who doesn't ;-) to find out how my unit tests help me to make sure
      my optimizations are ok. I still have to use the debugger more than
      I'd like to. Perhaps it's because I still have to "master" TDD, but the
      feeling in my gut tells me that it will stay dependent on the environment.

      Cheers,
      Manuel

      On 8/28/07, John Roth <JohnRoth1@...> wrote:
      >
      >
      > ----- Original Message -----
      > From: "Manuel Klimek" <klimek@... <klimek%40box4.net>>
      > To: <extremeprogramming@yahoogroups.com<extremeprogramming%40yahoogroups.com>
      > >
      > Sent: Tuesday, August 28, 2007 2:04 PM
      > Subject: Re: [XP] The relationship between coding effort and fixing effort
      >
      > > Hi John,
      > >
      > > thanks for your thoughts. I don't think that the formula must cover
      > /all/
      > > errors, but that it covers the errors from a certain point onwards.
      > > The trade-off is always
      > >
      > > | more up-front-effort vs. defects that are found later and have to be
      > > fixed. |
      > >
      > > This is covered by the usage of the defect rate. For example,
      > > if I wanted to check if a tester (in person) is cost-efficient with
      > > regards to the defect rate, I'd use the defect rate at the point where
      > > I want to inject the tester. All defects that are found /before/ this
      > > point in time are not included in the equation, because the tester
      > > would not be able to have an effect on them.
      > >
      > > If I want to measure a practice I want to include only the errors
      > > that can be found by that practice - which is than in the defect
      > > rate as well as in the estimated fixing cost per defect.
      > >
      > > Does that make sense?
      >
      > Sure. However, remember that experianced TDD practitioners
      > simply don't do a significant amount of debugging. As in many
      > of them haven't used the debugger in months if not longer.
      >
      > So TDD is replacing _all_ of the testing effort that would
      > otherwise be done by the developer. That's what many people
      > find difficult to accept. You're swapping the time spent on
      > (usually) unstructured testing and debugging for writing real
      > unit tests.
      >
      > Using the debugger to vet code is simply part of the programming
      > process for most developers; many of them don't have a good
      > idea of the amount of time they spend on unstructured testing and
      > debugging versus actually writing code.
      >
      > John Roth
      >
      >
      > >
      > > Cheers,
      > > Manuel
      > >
      > > On 8/28/07, John Roth <JohnRoth1@... <JohnRoth1%40gmail.com>>
      > wrote:
      > >>
      > >>
      > >>
      > >>
      > >>
      > >>
      > >> I'd say that the "defects per KLOC" is unrealistically low.
      > >> 20 defects per KLOC is usually the result of a significant
      > >> (albeit unstructured) testing effort.
      > >>
      > >> The number you want for your model is the number of
      > >> defects prior to the first compile, and also before any
      > >> desk checking.
      > >>
      > >> You also want to include the time spent on structured or
      > >> unstructured testing and integration.
      > >>
      > >> John Roth
      > >>
      > >>
      > >> ----- Original Message -----
      > >> From: "Manuel Klimek" <klimek@... <klimek%40box4.net>>
      > >> To: <extremeprogramming@yahoogroups.com<extremeprogramming%40yahoogroups.com>
      > >
      > >> Sent: Tuesday, August 28, 2007 1:06 PM
      > >> Subject: [XP] The relationship between coding effort and fixing effort
      > >>
      > >> > Hi,
      > >> >
      > >> > after I did a poll with my colleagues about whether they perceive TDD
      > >> > and PP to make them more productive (half of them thinks: nope), I
      > did
      > >> > some mathematical modeling trying to figure what the relationship
      > >> > between coding effort and bugfixing effort could be, just to get a
      > >> > grasp of it somehow ( *blog-entry warning* ;-)
      > >> > http://klimek.box4.net/blog/2007/08/28/defects-on-sale/
      > >> >
      > >> > I come to the conclusion that if you have a defect rate of 20 defects
      > >> > per KLOC and want to introduce a practice that doubles the coding
      > >> > effort and finds a tenth of the errors, the expected cost per error
      > >> > would have to be 500 times the cost for one line of code.
      > >> >
      > >> > I think this to be a worst-case like kind of calculation, but I don't
      > >> > really have a lot of experience with those management-consultantic
      > >> > numbers (and google is rather silent on those, I fear).
      > >> >
      > >> > Yep, I know that it's a lot of oversimplification, but I'd like to
      > >> > know: do you think this is a lot? What would you expect as effect
      > with
      > >> > regards to the defect rate when introducing PP or TDD? (just for
      > >> > reference: I know that TDD and PP have different effects besides
      > >> > improving the defect rate, but /if/ improving the defect rate already
      > >> > makes them cost efficient, it would be an interesting observation).
      > >> >
      > >> > Cheers,
      > >> > Manuel
      > >> >
      > >> > --
      > >> > http://klimek.box4.net
      > >> >
      > >>
      > >>
      > >>
      > >
      > >
      > >
      > > --
      > > http://klimek.box4.net
      > >
      >
      >
      >



      --
      http://klimek.box4.net


      [Non-text portions of this message have been removed]
    Your message has been successfully submitted and would be delivered to recipients shortly.