Loading ...
Sorry, an error occurred while loading the content.

RE: [scrumdevelopment] Reduces Or Moves Cost Of Change? (was RE: flattened cost of change curve)

Expand Messages
  • Tom Stambaugh
    I wonder if some of the apparently reduced cost of change in XP is instead an illusion resulting from moving those costs to the test development area - and
    Message 1 of 2 , Sep 30, 2003
    • 0 Attachment
      I wonder if some of the apparently reduced cost of change in XP is instead
      an illusion resulting from moving those costs to the test development area -
      and then not measuring them as carefully. When we calculate the "cost of
      change", do we include the cost of maintaining or extending the test
      suite/harness in our calculations?

      Could some of the practitioners perhaps contribute a brief summary of how
      they measure this "cost of change", and what costs are included? For
      example - do we include the cost of enumerating the user stories, then
      collecting and structuring the example data used to generate test cases in
      our cost summaries? Do we include the cost of refactoring the test harnesses
      when a new story or bug reveals fundamental errors or omissions?

      Perhaps because agile methods are still new to me, I find my head-scratching
      time often moves from the code under test to the test harness, rather than
      disappearing altogether. I find that an important part of my development
      time for new functionality is determining how to think about it. I have
      always rejected waterfall-style "white-board engineering" or "word
      engineering", in favor of Smalltalk-style prototyping. I'm not sure whether
      or how much XP has improved my prototyping - but I'm pretty sure it has NOT
      been exponential. Perhaps this falls under the umbrella of "spike
      solutions"?

      For example, I'm just rediscovering the power of streams, sequences, and
      delayed evaluation as described in the Wizard Book, because I'm working on a
      project where I'm manipulating LOTS of hierarchical structures. I can write
      the code (often just transcribing it from the Scheme examples in the text)
      easily enough - but how do I even think about *testing* it? Do I bother?

      I wonder how the *total* cost of new functionality (including
      headscratching, prototyping, and test development time) using XP compares
      to, for instance, my habit of prototyping and iterating until it works "good
      enough". I ask this because I find myself just as able to get lost in
      designing and coding unit tests as I used to get doing "prototypes".

      Thanks,
      Tom



      ----- Original Message -----
      From: "Steven Gordon" <sagordon@...>
      To: <scrumdevelopment@yahoogroups.com>
      Sent: Tuesday, September 30, 2003 12:06 PM
      Subject: RE: [scrumdevelopment] Re: flattened cost of change curve


      > Alistair,
      >
      > This still does not address my earlier observation:
      >
      > 1. This analysis addresses mistaken or misunderstood or misimplemented
      requirements -the much shorter feedback mechanisms of XP and other agile
      software development methodologies catch these kinds of problems much
      sooner, vastly shortening the time over which the exponential costs
      accelerate.
      >
      > 2. This analysis does not address the cost of changes not due to mistakes,
      but due to evolving requirements, new requirements, market changes, etc.
      >
      > Refactoring and simple design are a very minor contributor in the first
      case compared to time lag, but anecdotally they give XP a huge advantage
      over phasist approaches in dealing with the second kinds of changes. If the
      change curve is also exponential for this second type of changes, then how
      does refactoring and simple design lessen their costs? Or do they really?
      >
      > Steven Gordon
      >
      > -----Original Message-----
      > From: acockburn@... [mailto:acockburn@...]
      > Sent: Tue 9/30/2003 8:30 AM
      > To: scrumdevelopment@yahoogroups.com
      > Cc:
      > Subject: Re: [scrumdevelopment] Re: flattened cost of change curve
      >
      > I find it non-stop fascinating that people are so willing to say something
      > that is exactly wrong, and cover it over by saying, "but let's not get
      into fine
      > details about it." I find this happens in discussions of Liskov
      Substitution
      > Principle and the exponential cost of change curve. Both are mathematical,
      and
      > therefore are Right or Wrong.
      >
      > Barry Boehm declared that the cost of finding and fixing a mistake grows
      > expentially as it remains undetected through requirements, coding, testing
      and
      > deployment. Kent wrote: "...the curve is no longer valid. Or rather, with
      a
      > combination of technology and programming practices, it is possible to
      experience a
      > curve that is really quite the opposite." and then drew a log(n) curve.
      >
      > Both Barry and Kent are talking math here. The math is right or wrong. If
      > someone chooses to ask the question: "Is the curve still exponential?"
      then the
      > answer is Yes or No, and not "Not really, because there are lots of little
      > exponential curves, so it's not really exponential."
      >
      > I wrote up and Ron published my defense of it being exponential even in
      XP. I
      > stand by those arguments.
      >
      > Ilya asks: <<the thing XP calls into question, as I understand it, is that
      > "bugs" introduced in the *requirements* are much easier to fix (when
      > working XP-style). That's why we don't try to get them all "correct"
      > before starting with implementation.... What do you think?>>
      >
      > You can see the analysis in the article, but the main point for Ilya's
      > question is that he is only comparing requirements and coding. He left out
      the next
      > two steps - acceptance testing by the customer and field deployment. I
      don't
      > think there are yet any XP advocates who argue that we shouldn't bother to
      get
      > requirements or code correct because it is so much easier to correct it in
      the
      > field. We hear quite the opposite: get the bugs out before deploying. Why?
      > Because it is so much more expensive. The reason requirements can overlap
      > coding has to do with cost of communications (still inside the exponential
      curve),
      > and the value of feedback from coding to requirements (a different cost
      factor
      > altogether, and worthy of its own scrutiny).
      >
      > Ron correctly writes: <<The biggest question may be whether one seeks to
      > understand what is
      > meant by the notion of flattening the change curve,>>
      >
      > He is correct -- and we don't gain that understanding by pretending an
      > exponential curve is logarithmic and then working to explain away the
      funny mismatch
      > in shape, but rather by first establishing which curve we are dealing with
      > and then mapping that back to project life, and back again to math.
      >
      > I already toook my stab at this, and I stand by my conclusion in the
      absence
      > of counter data or a better explanation:
      > <<
      > "Bureaucratic Specification (BS) methodologies . . .also try
      > to handle the exponential growth, but basically their cost for everything
      is
      > a Very Large constant greater than XP's. So much greater that XP is
      running
      > the cost-to-fix in "p" (programming time) while the BS methodologies are
      > still running in "r" (requirements time). So XP has the program written
      and
      > fixed by the time the BS methodology has just the spec written and
      studied.
      > Keep the exponential curve, but drop everything by very large constants,
      and
      > XP is likely to have shipped the product while the other one is still
      being
      > designed.
      > >>
      >
      > Given that Ron has already agreed that the curve is exponential, can we
      stop
      > trying to pretend it isn't, and get on with the "understanding" part?
      >
      > Alistair
      >
      >
      >
      >
      >
      > To Post a message, send it to: scrumdevelopment@...
      > To Unsubscribe, send a blank message to:
      scrumdevelopment-unsubscribe@...
      >
      > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
      >
      >
    • Ron Jeffries
      ... I doubt whether anyone has done (or could do) a comparison, as no two projects are comparable. However, those of who do more testing than we used to are
      Message 2 of 2 , Sep 30, 2003
      • 0 Attachment
        On Tuesday, September 30, 2003, at 12:12:59 PM, Tom Stambaugh wrote:

        > I wonder if some of the apparently reduced cost of change in XP is instead
        > an illusion resulting from moving those costs to the test development area -
        > and then not measuring them as carefully. When we calculate the "cost of
        > change", do we include the cost of maintaining or extending the test
        > suite/harness in our calculations?

        I doubt whether anyone has done (or could do) a comparison, as no two
        projects are comparable.

        However, those of who do more testing than we used to are quite sure that
        our total time for development is reduced thereby, and that conformance to
        requirements is increased.

        > Could some of the practitioners perhaps contribute a brief summary of how
        > they measure this "cost of change", and what costs are included? For
        > example - do we include the cost of enumerating the user stories, then
        > collecting and structuring the example data used to generate test cases in
        > our cost summaries? Do we include the cost of refactoring the test harnesses
        > when a new story or bug reveals fundamental errors or omissions?

        >From this list, I'd note:

        - All the information in the user stories must be created in any case,
        and there is no reason to think that doing it in story style is more
        costly. If anything, owing to reduced formality, I would expect it to be
        less.

        - Unless we are comparing a tested project to an untested one, all the
        information used for tests needs to be created somewhere in the project
        context in any case. Are you thinking that perhaps agile projects test
        "too much"?

        - <sigh> Refactoring as part of incremental development is not a matter
        of doing large amounts of work over. It is more like sanding a surface
        prior to painting. First the coarse sandpaper takes down the rough spots,
        then finer and finer grades until the surface is as smooth as we want.
        People who sand surfaces don't think of sanding multiple times as doing
        things over again. Neither should people who develop software, or test
        harnesses, incrementally, think of refactoring as doing things over
        again. It's doing the thing first roughly, then refining it.

        > Perhaps because agile methods are still new to me, I find my head-scratching
        > time often moves from the code under test to the test harness, rather than
        > disappearing altogether.

        I may not be correctly apprehending your use of the word "harness".
        Certainly one thinks about testing. If one is spending much time thinking
        about the harness in the sense of framework, it might be too much. We don't
        need a testing framework, we need tests.

        More thinking about the tests seems to reduce the time spend coding by more
        than the same amount.

        > I find that an important part of my development
        > time for new functionality is determining how to think about it. I have
        > always rejected waterfall-style "white-board engineering" or "word
        > engineering", in favor of Smalltalk-style prototyping. I'm not sure whether
        > or how much XP has improved my prototyping - but I'm pretty sure it has NOT
        > been exponential. Perhaps this falls under the umbrella of "spike
        > solutions"?

        I'm not sure that XP claims to improve anything exponentially, most
        especially not a good evolution-from-prototype approach. What XP claims is
        that the old saw about the cost of change being exponential gets translated
        in the minds of people as "change is bad, very bad", and that in fact, in
        incremental design supported by tests and refactoring, change is not all
        that bad after all.

        > For example, I'm just rediscovering the power of streams, sequences, and
        > delayed evaluation as described in the Wizard Book, because I'm working on a
        > project where I'm manipulating LOTS of hierarchical structures. I can write
        > the code (often just transcribing it from the Scheme examples in the text)
        > easily enough - but how do I even think about *testing* it? Do I bother?

        I don't know. The first time I spent more than five minutes in the
        debugger, I'd start trying to figure it out.

        > I wonder how the *total* cost of new functionality (including
        > headscratching, prototyping, and test development time) using XP compares
        > to, for instance, my habit of prototyping and iterating until it works "good
        > enough". I ask this because I find myself just as able to get lost in
        > designing and coding unit tests as I used to get doing "prototypes".

        It could happen. XP and Scrum and so on are probably more about taming the
        effects of change, and making results more predictable thereby, than about
        reducing the overall cost of projects. We get projects done that would
        otherwise not get done; we provide information to enable knowing when
        things will be done; and we generally produce results that satisfy the
        customers.

        If you're already doing those things, XP or Scrum might help you do them
        more easily, but Done At Right Quality As Predicted is what it's really all
        about.

        Ron Jeffries
        www.XProgramming.com
        No one expects the Spanish Inquisition ...
      Your message has been successfully submitted and would be delivered to recipients shortly.