Loading ...
Sorry, an error occurred while loading the content.

Re: [scrumdevelopment] flattened cost of change curve

Expand Messages
  • Stephen Haberman
    On Sat, Sep 27, 2003 at 05:47:50PM -0400, Ron Jeffries wrote: [snip] ... Makes sense. ... Understood; thanks for giving me your thoughts on the subject. I ll
    Message 1 of 30 , Sep 27, 2003
    • 0 Attachment
      On Sat, Sep 27, 2003 at 05:47:50PM -0400, Ron Jeffries wrote:

      [snip]

      > I believe that the curve effect is more or less present in any
      > iterative method. Details matter: refactoring in particular.

      Makes sense.

      > It's not a simple question with a simple answer.

      Understood; thanks for giving me your thoughts on the subject.

      I'll clear my head for a day or so, then re-read the chapter in XP
      Explained, Alistair's article, and this thread, and hopefully come
      out understanding the nuances a bit more.

      - Stephen
    • Steven Gordon
      On Sat, Sep 27, 2003 at 05:47:50PM -0400, Ron Jeffries wrote: [snip] ... [snip] I believe this discussion is confounding two related, but distinct phenomena:
      Message 2 of 30 , Sep 27, 2003
      • 0 Attachment
        On Sat, Sep 27, 2003 at 05:47:50PM -0400, Ron Jeffries wrote:

        [snip]

        > I believe that the curve effect is more or less present in any
        > iterative method. Details matter: refactoring in particular.

        [snip]

        I believe this discussion is confounding two related, but distinct phenomena:
        1. The cost of mistakes
        2. The cost of change

        Cost of Mistakes

        The curve effect of multiple shorter exponential curves is the cost of mistakes. In a phasist process, some mistakes are likely to not be caught until the customer has the chance to experience the implementation. The long time delay between when the mistake was made (e.g., during big up front analysis or design) makes the exponential cost huge. In an iterative process where individual features are developed from the customer requirement to implementation in a short time period, these mistakes are caught more quickly, making the exponential cost much much less.

        I do not believe refactoring or even expert OO skills has a great effect on this cost. The effect is almost entirely due to the significantly shorter time between starting the work and getting customer feedback.

        Cost of Change

        On the other hand, cost of change is the cost of reworking the code to handle a new requirement that is in conflict with the work already done on the system (even if the work already done was totally correct given the previously know requirements).

        BDUF proponents may argue that it is less costly to apply such changes if the implementation has not yet started - just trace the impact of the change via the mapping from requirements to design artifacts and make the necessary changes. XP argues that its cost of change is close to linear due to the application of simple OO design and refactoring to keep that design simple. I do not believe that this issue can be understood as the decomposition of a single long exponential curve into a series of short exponential curves.

        Steven Gordon





        The cost of change
      • Tom Stambaugh
        The point that I got from Alastair s paper, whether or not he intended it, is that the familiar curve from Explained is perhaps not the absolute cost over
        Message 3 of 30 , Sep 27, 2003
        • 0 Attachment
          The point that I got from Alastair's paper, whether or not he intended it,
          is that the familiar curve from "Explained" is perhaps not the absolute cost
          over time, but instead a "phase space" diagram - the cost of a given change
          in XP vs the same curve in "conventional" approaches. To paraphrase his
          summary, as the cost of a given change gets higher in higher in a
          "bureaucratic specification" approach, the cost of the same change is much
          lower in XP. I like his conclusion that the curve more accurately describes
          the *phase space* comparison of the two approaches.

          I also agree with Alastair's speculation that the shape of the curve is
          qualitative, rather than quantitative.

          Thanks,
          Tom
        • aacockburn
          (third time I m attempting to post this) ... Ron: Honestly I can t figure out what Alistair s point was. He was mostly trying to show that the curve was still
          Message 4 of 30 , Sep 29, 2003
          • 0 Attachment
            (third time I'm attempting to post this)
            Stephen:
            > Alistair thought the curve is still exponential, XP just handles
            > change better (by fixing code better and faster) so it has smaller
            > constants and grows at a slower pace when compared to the
            > high-bureaucracy expotential curve.

            Ron:
            Honestly I can't figure out what Alistair's point was. He
            was mostly trying to show that the curve was still exponential,
            and whatever the larger point was, I missed it.

            Stephen:
            > You're saying by XP handling change better, it does in fact change
            > the curve to be logarithmic, as (perhaps due to the good engineering
            > practices) the next change gets to start the curve all over again,
            > there by keeping the curve from ever reaching a rapidly growing
            > rate.

            Ron:
            No, I'm just saying that it remains exponential in T, but that
            changes tend
            to be limited to low values of T.
            --->
            Hi, Ron,
            I'm really glad that you got the point of the article, however
            much you deny it.
            My point was to demonstrate that the curve is still exponential.
            There was no larger point -- that was exactly the point that
            was under controversy at the time.
            The point of the article was to argue that the curve is
            still exponential.
            From your last posting, it seems you think so to.
            Congrats
            (I hope that now you can indeed figure out what Alistair's point was.)

            Alistair

            ==============================================
            Alistair Cockburn
            President, Humans and Technology
            Program Director, Agile Development Conference
            (http://AgileDevelopmentConference.com)

            http://alistair.cockburn.us alistair.cockburn@...
            1814 E. Fort Douglas Circle, Salt Lake City, UT 84103
            Phone: 801.582-3162 Fax: 775.416.6457

            Author of
            "Surviving Object-Oriented Projects" (1998)
            "Writing Effective Use Cases" (Jolt Productivity Award 2001)
            "Agile Software Development" (Jolt Productivity Award 2002)

            "La perfection est atteinte non quand il ne reste rien a ajouter,
            mais quand il ne reste rien a enlever." (Saint-Exupery)
            ==============================================
          • aacockburn
            ... Jeffries: No, I don t think he thought that. I think he thought that XP flattens the curve, ... The white book says: (p.22) The problem is that the curve
            Message 5 of 30 , Sep 29, 2003
            • 0 Attachment
              Stephen:
              > Beck thought XP, with its good engineering practices, changes the
              > curve to be logarithmic.

              Jeffries:
              No, I don't think he thought that. I think he thought that XP
              flattens the
              curve,

              --->
              The white book says: (p.22)
              "The problem is that the curve is no longer valid. Or rather, with a
              combination of technology and programming practices, it is possible
              to experience a curve that is really quite the opposite."

              Kent's writing supports Stephen's view

              ==============================================
              Alistair Cockburn
              President, Humans and Technology
              Program Director, Agile Development Conference
              (http://AgileDevelopmentConference.com)

              http://alistair.cockburn.us alistair.cockburn@...
              1814 E. Fort Douglas Circle, Salt Lake City, UT 84103
              Phone: 801.582-3162 Fax: 775.416.6457

              Author of
              "Surviving Object-Oriented Projects" (1998)
              "Writing Effective Use Cases" (Jolt Productivity Award 2001)
              "Agile Software Development" (Jolt Productivity Award 2002)

              "La perfection est atteinte non quand il ne reste rien a ajouter,
              mais quand il ne reste rien a enlever." (Saint-Exupery)
              ==============================================
            • aacockburn
              In a message dated 9/28/2003 8:28:07 AM Mountain Daylight Time, Ron ... No, I don t think he thought that. I think he thought that XP flattens the curve, ...
              Message 6 of 30 , Sep 29, 2003
              • 0 Attachment
                In a message dated 9/28/2003 8:28:07 AM Mountain Daylight Time, Ron
                Jeffries writes:

                > Beck thought XP, with its good engineering practices, changes the
                > curve to be logarithmic.

                No, I don't think he thought that. I think he thought that XP
                flattens the
                curve,
                --->
                Not only that, but it is very interesting to look at his white book
                again in the context of modern XP ...
                p.22:
                " 'The cost to fix a problem in a piece of software rises
                exponentially over time.' ... I resolved then and there that I would
                never let a problem get through to production. ..."

                It seems that Kent has come full circle. One of the big ads for XP
                these days is exactly that it keeps problems from getting through to
                production ! Why? Well, because it is so much more costly to fix if
                it gets through. Therefore, please do pair programming, automated
                testing, tiny changes, etc etc.

                I thank Tom Stanbaugh for finding the relevant quote in my article
                from some years ago (anyone know which year that got written?) which
                is now in play:

                "The exponential cost curve is mostly in detecting and
                communicating the Mistake and naming the change that is
                to be made. XP cannot change that curve, and indeed,
                XP takes that increasing cost curve neatly into account.
                So the first lesson I get is that one should not base a
                defense of XP on the ABSENCE of the curve, but rather on
                the PRESENCE of the exponential cost curve."

                At this point, the exponential cost of change is still in full play
                and is part of XP's standard sales pitch.

                ==============================================
                Alistair Cockburn
                President, Humans and Technology
                Program Director, Agile Development Conference
                (http://AgileDevelopmentConference.com)

                http://alistair.cockburn.us alistair.cockburn@...
                1814 E. Fort Douglas Circle, Salt Lake City, UT 84103
                Phone: 801.582-3162 Fax: 775.416.6457

                Author of
                "Surviving Object-Oriented Projects" (1998)
                "Writing Effective Use Cases" (Jolt Productivity Award 2001)
                "Agile Software Development" (Jolt Productivity Award 2002)

                "La perfection est atteinte non quand il ne reste rien a ajouter,
                mais quand il ne reste rien a enlever." (Saint-Exupery)
                ==============================================
              • Mike Beedle
                Alistair accurately stated: The exponential cost curve is mostly in detecting and communicating the Mistake and naming the change that is to be made. XP
                Message 7 of 30 , Sep 29, 2003
                • 0 Attachment
                  Alistair accurately stated:
                  "The exponential cost curve is mostly in detecting and
                  communicating the Mistake and naming the change that is
                  to be made. XP cannot change that curve, and indeed,
                  XP takes that increasing cost curve neatly into account.
                  So the first lesson I get is that one should not base a
                  defense of XP on the ABSENCE of the curve, but rather on
                  the PRESENCE of the exponential cost curve."


                  Alistair,

                  Well said.

                  If bugs are our software's enemies then ...

                  ... the highest form of generalship is to
                  balk the enemy's plans (in UNIT TESTING and ACCEPTANCE
                  TESTING) ; the next best is to prevent
                  the junction of the enemy's forces (in SYSTEM TESTING);
                  the next in order is to attack the enemy's army in the field
                  (in early PRODUCTION); and the worst policy of all is to besiege
                  walled cities (Here the bugs have taken over
                  our software and have made cities within it, where they
                  reside comfortably, while we struggle and fight them
                  in MAINTENANCE .... )

                  from
                  THE ART OF (SOFTWARE) WAR
                  III. ATTACK BY STRATAGEM
                  SUN TZU

                  - Mike
                • Ron Jeffries
                  ... Nice ... Ron Jeffries www.XProgramming.com No one expects the Spanish Inquisition ...
                  Message 8 of 30 , Sep 29, 2003
                  • 0 Attachment
                    On Monday, September 29, 2003, at 10:52:25 PM, Mike Beedle wrote:

                    > If bugs are our software's enemies then ...

                    > ... the highest form of generalship is to
                    > balk the enemy's plans (in UNIT TESTING and ACCEPTANCE
                    > TESTING) ; the next best is to prevent
                    > the junction of the enemy's forces (in SYSTEM TESTING);
                    > the next in order is to attack the enemy's army in the field
                    > (in early PRODUCTION); and the worst policy of all is to besiege
                    > walled cities (Here the bugs have taken over
                    > our software and have made cities within it, where they
                    > reside comfortably, while we struggle and fight them
                    > in MAINTENANCE .... )

                    > from
                    > THE ART OF (SOFTWARE) WAR
                    > III. ATTACK BY STRATAGEM
                    > SUN TZU

                    Nice ...

                    Ron Jeffries
                    www.XProgramming.com
                    No one expects the Spanish Inquisition ...
                  • Mike Beedle
                    ... Hmm, I would expect XPers to say :-) I prefer a new edition of the Spanish Inquisition than to ever let .... untested bugs in my App!!!! paraphrasing
                    Message 9 of 30 , Sep 29, 2003
                    • 0 Attachment
                      Ron wrote:
                      > No one expects the Spanish Inquisition ...


                      Hmm, I would expect XPers to say :-)

                      I prefer a new edition of the Spanish Inquisition
                      than to ever let .... untested bugs in my App!!!!

                      paraphrasing Professor Higgins, in My Fair "App" Lady

                      Of course, testing would be minimized to verify "what"
                      the software did if we only wrote "provable" software,

                      - Mike

                      -- quicksort in Curry
                      filter_ :: (a -> Bool) -> [a] -> [a];
                      filter_ _ [] = []
                      filter_ p (x:xs) = if p x then x : filter_ p xs
                      else filter_ p xs

                      qsort :: [Int] -> [Int]
                      qsort [] = []
                      qsort (x:l) = qsort (filter (< x) l) x : qsort (filter (>= x) l)
                    • Ilja Preuss
                      ... In fact, I don t think it s in full play! You are right that XP takes care of the fact that a bug introduced in implementation is much harder to fix the
                      Message 10 of 30 , Sep 30, 2003
                      • 0 Attachment
                        aacockburn wrote:

                        > At this point, the exponential cost of change is still in full play
                        > and is part of XP's standard sales pitch.

                        In fact, I don't think it's in full play!

                        You are right that XP takes care of the fact that a bug introduced in
                        implementation is much harder to fix the later it is found. Therefore
                        the early and excessive testing.

                        *But* - the thing XP calls into question, as I understand it, is that
                        "bugs" introduced in the *requirements* are much easier to fix (when
                        working XP-style). That's why we don't try to get them all "correct"
                        before starting with implementation.

                        What do you think?

                        Curious, Ilja

                        --
                        ++ Business Intelligence: disy Cadenza Web - drei Klicks zum Ergebnis
                        ++ http://www.disy.net/cadenza
                        ++ disy Conference: Telefonkonferenz mit
                        ++ integrierter Datenkonferenz http://www.disy.net/conference ++

                        Ilja Preuß preuss@...
                        disy Informationssysteme GmbH http://www.disy.net
                        Stephanienstr. 30 Tel: +49 721 1 600 624
                        D-76133 Karlsruhe, Germany Fax: +49 721 1 600 605

                        ++ disy: Kosteneinsparungen durch Optimierung Ihrer IuK-Prozesse ++
                      • Brad Appleton
                        ... I think XP challenges all assertions predicated upon a presumption of a waterfall-lifecycle as XP challenges the waterfall-lifecycle itself and the very
                        Message 11 of 30 , Sep 30, 2003
                        • 0 Attachment
                          > *But* - the thing XP calls into question, as I understand it, is that
                          > "bugs" introduced in the *requirements* are much easier to fix (when
                          > working XP-style). That's why we don't try to get them all "correct"
                          > before starting with implementation.

                          I think XP challenges all assertions predicated upon a
                          presumption of a waterfall-lifecycle as XP challenges the
                          waterfall-lifecycle itself and the very notion of separable
                          "phases" of isolated activity-flow that focusing on producing
                          intermediate artifacts, as opposed to a single inseparable
                          flow of collaborative activities that focuses on producing
                          working software in "small enough" increments with high frequency.

                          XP says "what happens to all these waterfall-based presumptions
                          about cost-of-change of I go to the extreme of taking the
                          "limit" as the size of the delta (change) approaches zero?" Then
                          it is still true that the longer the period of time in between
                          fault injection and fault detection, the exponentially higher
                          the cost of fault-correction, because the more work that was
                          done that must be reworked.

                          But it is no longer true to make the phase-based assumption
                          that says 'the the lifecycle phase in which it was found, the
                          exponentially higher the cost to fix it. Because the delta is
                          so small and the phases are so much smaller and statistically
                          insignificant that it becomes a greater interruption of flow
                          to focus on intermediate artifacts and formal reviews for the
                          amount of work done in a single episode or feedback-cycle than
                          to rely upon the "in flow" feedback-loops in XP. And the amount
                          of work done between feedback cycles is whittled down so small,
                          that if you find it even only after you integrate your changes
                          and test them, it is still such a small unit of work that the
                          amount of rework is comparable rather than exponential.

                          I think that's what is meant by "flattening" the change curve
                          by taking the limit as delta-size approaches zero to make the
                          feedback loops be collaborative and/or in-flow activities rather
                          than separate across phase boundaries in distinct activity flows.
                          And that focusing on the whole via piecemeal growth works better
                          than the waterfall-based "magnum opus" divide-and-conquer approach
                          that deconstructs and separates to focus more on the parts and
                          their sum than on the synergistic whole.

                          Just my opinion ;-)

                          --
                          Brad Appleton <brad@...> www.bradapp.net
                          Software CM Patterns (www.scmpatterns.com)
                          Effective Teamwork, Practical Integration
                          "And miles to go before I sleep." -- Robert Frost
                        • Ron Jeffries
                          ... Yes. The biggest question may be whether one seeks to understand what is meant by the notion of flattening the change curve, or merely to refute it. Ron
                          Message 12 of 30 , Sep 30, 2003
                          • 0 Attachment
                            On Tuesday, September 30, 2003, at 10:51:05 AM, Brad Appleton wrote:

                            > I think that's what is meant by "flattening" the change curve
                            > by taking the limit as delta-size approaches zero to make the
                            > feedback loops be collaborative and/or in-flow activities rather
                            > than separate across phase boundaries in distinct activity flows.
                            > And that focusing on the whole via piecemeal growth works better
                            > than the waterfall-based "magnum opus" divide-and-conquer approach
                            > that deconstructs and separates to focus more on the parts and
                            > their sum than on the synergistic whole.

                            Yes. The biggest question may be whether one seeks to understand what is
                            meant by the notion of flattening the change curve, or merely to refute it.

                            Ron Jeffries
                            www.XProgramming.com
                            The opinions expressed here /are/ necessarily those of XProgramming.com.
                            But I might change my mind.
                          • acockburn@aol.com
                            I find it non-stop fascinating that people are so willing to say something that is exactly wrong, and cover it over by saying, but let s not get into fine
                            Message 13 of 30 , Sep 30, 2003
                            • 0 Attachment
                              I find it non-stop fascinating that people are so willing to say something that is exactly wrong, and cover it over by saying, "but let's not get into fine details about it." I find this happens in discussions of Liskov Substitution Principle and the exponential cost of change curve. Both are mathematical, and therefore are Right or Wrong.
                               
                              Barry Boehm declared that the cost of finding and fixing a mistake grows expentially as it remains undetected through requirements, coding, testing and deployment. Kent wrote: "...the curve is no longer valid. Or rather, with a combination of technology and programming practices, it is possible to experience a curve that is really quite the opposite." and then drew a log(n) curve.
                               
                              Both Barry and Kent are talking math here. The math is right or wrong. If someone chooses to ask the question: "Is the curve still exponential?" then the answer is Yes or No, and not "Not really, because there are lots of little exponential curves, so it's not really exponential." 
                               
                              I wrote up and Ron published my defense of it being exponential even in XP. I stand by those arguments.
                               
                              Ilya asks: <<the thing XP calls into question, as I understand it, is that
                              "bugs" introduced in the *requirements* are much easier to fix (when
                              working XP-style). That's why we don't try to get them all "correct"
                              before starting with implementation.... What do you think?>>
                               
                              You can see the analysis in the article, but the main point for Ilya's question is that he is only comparing requirements and coding. He left out the next two steps - acceptance testing by the customer and field deployment. I don't think there are yet any XP advocates who argue that we shouldn't bother to get requirements or code correct because it is so much easier to correct it in the field. We hear quite the opposite: get the bugs out before deploying. Why? Because it is so much more expensive.  The reason requirements can overlap coding has to do with cost of communications (still inside the exponential curve), and the value of feedback from coding to requirements (a different cost factor altogether, and worthy of its own scrutiny).
                               
                              Ron correctly writes: <<The biggest question may be whether one seeks to understand what is
                              meant by the notion of flattening the change curve,>>
                               
                              He is correct -- and we don't gain that understanding by pretending an exponential curve is logarithmic and then working to explain away the funny mismatch in shape, but rather by first establishing which curve we are dealing with and then mapping that back to project life, and back again to math.
                               
                              I already toook my stab at this, and I stand by my conclusion in the absence of counter data or a better explanation:
                              <<
                              "Bureaucratic Specification (BS) methodologies . . .also try
                              to handle the exponential growth, but basically their cost for everything is
                              a Very Large constant greater than XP's. So much greater that XP is running
                              the cost-to-fix in "p" (programming time) while the BS methodologies are
                              still running in "r" (requirements time). So XP has the program written and
                              fixed by the time the BS methodology has just the spec written and studied.
                              Keep the exponential curve, but drop everything by very large constants, and
                              XP is likely to have shipped the product while the other one is still being
                              designed.

                              >>
                               
                              Given that Ron has already agreed that the curve is exponential, can we stop trying to pretend it isn't, and get on with the "understanding" part?
                               
                              Alistair
                               
                            • Ron Jeffries
                              ... Paraphrasing from a Wendy s commercial that I like: Your concept of Right or Wrong in human discourse: very narrow. Ron Jeffries www.XProgramming.com XP
                              Message 14 of 30 , Sep 30, 2003
                              • 0 Attachment
                                On Tuesday, September 30, 2003, at 11:30:27 AM, acockburn@... wrote:

                                > I find it non-stop fascinating that people are so willing to say something
                                > that is exactly wrong, and cover it over by saying, "but let's not get into fine
                                > details about it." I find this happens in discussions of Liskov Substitution
                                > Principle and the exponential cost of change curve. Both are mathematical, and
                                > therefore are Right or Wrong.

                                Paraphrasing from a Wendy's commercial that I like:

                                Your concept of Right or Wrong in human discourse: very narrow.

                                Ron Jeffries
                                www.XProgramming.com
                                XP says: Don't just sit on your DUF, do something. Get some feedback.
                              • acockburn@aol.com
                                In a message dated 9/30/2003 9:47:00 AM Mountain Daylight Time, ronjeffries@XProgramming.com writes:
                                Message 15 of 30 , Sep 30, 2003
                                • 0 Attachment
                                  In a message dated 9/30/2003 9:47:00 AM Mountain Daylight Time, ronjeffries@... writes:
                                  <<Paraphrasing from a Wendy's commercial that I like:
                                    Your concept of Right or Wrong in human discourse: very narrow.>>
                                  For better or worse (both, actually) mathematics doesn't count as "human" discourse in that sense :-)
                                   
                                  cheers
                                • Steven Gordon
                                  Alistair, This still does not address my earlier observation: 1. This analysis addresses mistaken or misunderstood or misimplemented requirements -the much
                                  Message 16 of 30 , Sep 30, 2003
                                  • 0 Attachment
                                    Alistair,

                                    This still does not address my earlier observation:

                                    1. This analysis addresses mistaken or misunderstood or misimplemented requirements -the much shorter feedback mechanisms of XP and other agile software development methodologies catch these kinds of problems much sooner, vastly shortening the time over which the exponential costs accelerate.

                                    2. This analysis does not address the cost of changes not due to mistakes, but due to evolving requirements, new requirements, market changes, etc.

                                    Refactoring and simple design are a very minor contributor in the first case compared to time lag, but anecdotally they give XP a huge advantage over phasist approaches in dealing with the second kinds of changes. If the change curve is also exponential for this second type of changes, then how does refactoring and simple design lessen their costs? Or do they really?

                                    Steven Gordon

                                    -----Original Message-----
                                    From: acockburn@... [mailto:acockburn@...]
                                    Sent: Tue 9/30/2003 8:30 AM
                                    To: scrumdevelopment@yahoogroups.com
                                    Cc:
                                    Subject: Re: [scrumdevelopment] Re: flattened cost of change curve

                                    I find it non-stop fascinating that people are so willing to say something
                                    that is exactly wrong, and cover it over by saying, "but let's not get into fine
                                    details about it." I find this happens in discussions of Liskov Substitution
                                    Principle and the exponential cost of change curve. Both are mathematical, and
                                    therefore are Right or Wrong.

                                    Barry Boehm declared that the cost of finding and fixing a mistake grows
                                    expentially as it remains undetected through requirements, coding, testing and
                                    deployment. Kent wrote: "...the curve is no longer valid. Or rather, with a
                                    combination of technology and programming practices, it is possible to experience a
                                    curve that is really quite the opposite." and then drew a log(n) curve.

                                    Both Barry and Kent are talking math here. The math is right or wrong. If
                                    someone chooses to ask the question: "Is the curve still exponential?" then the
                                    answer is Yes or No, and not "Not really, because there are lots of little
                                    exponential curves, so it's not really exponential."

                                    I wrote up and Ron published my defense of it being exponential even in XP. I
                                    stand by those arguments.

                                    Ilya asks: <<the thing XP calls into question, as I understand it, is that
                                    "bugs" introduced in the *requirements* are much easier to fix (when
                                    working XP-style). That's why we don't try to get them all "correct"
                                    before starting with implementation.... What do you think?>>

                                    You can see the analysis in the article, but the main point for Ilya's
                                    question is that he is only comparing requirements and coding. He left out the next
                                    two steps - acceptance testing by the customer and field deployment. I don't
                                    think there are yet any XP advocates who argue that we shouldn't bother to get
                                    requirements or code correct because it is so much easier to correct it in the
                                    field. We hear quite the opposite: get the bugs out before deploying. Why?
                                    Because it is so much more expensive. The reason requirements can overlap
                                    coding has to do with cost of communications (still inside the exponential curve),
                                    and the value of feedback from coding to requirements (a different cost factor
                                    altogether, and worthy of its own scrutiny).

                                    Ron correctly writes: <<The biggest question may be whether one seeks to
                                    understand what is
                                    meant by the notion of flattening the change curve,>>

                                    He is correct -- and we don't gain that understanding by pretending an
                                    exponential curve is logarithmic and then working to explain away the funny mismatch
                                    in shape, but rather by first establishing which curve we are dealing with
                                    and then mapping that back to project life, and back again to math.

                                    I already toook my stab at this, and I stand by my conclusion in the absence
                                    of counter data or a better explanation:
                                    <<
                                    "Bureaucratic Specification (BS) methodologies . . .also try
                                    to handle the exponential growth, but basically their cost for everything is
                                    a Very Large constant greater than XP's. So much greater that XP is running
                                    the cost-to-fix in "p" (programming time) while the BS methodologies are
                                    still running in "r" (requirements time). So XP has the program written and
                                    fixed by the time the BS methodology has just the spec written and studied.
                                    Keep the exponential curve, but drop everything by very large constants, and
                                    XP is likely to have shipped the product while the other one is still being
                                    designed.
                                    >>

                                    Given that Ron has already agreed that the curve is exponential, can we stop
                                    trying to pretend it isn't, and get on with the "understanding" part?

                                    Alistair
                                  • acockburn@aol.com
                                    In a message dated 9/30/2003 10:15:29 AM Mountain Daylight Time, sagordon@asu.edu writes: 2. This analysis does not address the cost of changes not due to
                                    Message 17 of 30 , Sep 30, 2003
                                    • 0 Attachment
                                      In a message dated 9/30/2003 10:15:29 AM Mountain Daylight Time, sagordon@... writes:
                                      2. This analysis does not address the cost of changes not due to mistakes, but due to evolving requirements, new requirements, market changes, etc.
                                      --->
                                       
                                      Correct. The exponential cost of change curve deals entirely with catching and fixing mistakes.
                                       
                                       
                                      Injecting a new requirement into the code has to do with (here comes my first guess):
                                      1. the weightiness of the process and
                                      2. the tidyness or simplicity of the design
                                      3. the availability of automated regression tests
                                       
                                      Weightier process take longer, quite simply.
                                      Simpler designs are easier and quicker to shift.
                                      Automated regression tests allow one to detect more quickly if some damage has gone to a different section of the system.
                                       
                                       
                                      Alistair
                                       
                                       
                                    • Edmund Schweppe
                                      ... I haven t got Kent s White Book in front of me, and I don t have Barry s paper anywhere, so I m speaking from oft-fragile memory here, but ... AFAIK, Barry
                                      Message 18 of 30 , Sep 30, 2003
                                      • 0 Attachment
                                        acockburn@... wrote:
                                        > I find it non-stop fascinating that people are so willing to say something
                                        > that is exactly wrong, and cover it over by saying, "but let's not get into fine
                                        > details about it." I find this happens in discussions of Liskov Substitution
                                        > Principle and the exponential cost of change curve. Both are mathematical, and
                                        > therefore are Right or Wrong.
                                        >
                                        > Barry Boehm declared that the cost of finding and fixing a mistake grows
                                        > expentially as it remains undetected through requirements, coding, testing and
                                        > deployment. Kent wrote: "...the curve is no longer valid. Or rather, with a
                                        > combination of technology and programming practices, it is possible to experience a
                                        > curve that is really quite the opposite." and then drew a log(n) curve.

                                        I haven't got Kent's White Book in front of me, and I don't have Barry's
                                        paper anywhere, so I'm speaking from oft-fragile memory here, but ...

                                        AFAIK, Barry Boehm's research showed the cost of *fixing defects* grew
                                        exponentially.

                                        AFAIK, Kent Beck's assertion is that the cost of *making changes* under
                                        XP does not grow exponentially.

                                        AFAIK, *fixing defects* != *making changes*. More specifically, there
                                        exist changes to software that do not involve fixing existing defects.
                                        (Obviously, fixing a defect involves making a change. Somewhere.)

                                        Thus, AFAIK, both gentlemen may be correct.

                                        Unfortunately, the exponential cost curve that Barry Boehm described has
                                        spent the last several decades labeled a cost of *change* curve, rather
                                        than a cost of *bugfix* curve. If it is in fact a cost of bugfix curve,
                                        then the "cost of change curve" could easily be dramatically less than
                                        exponential for agile methods which emphasize short iterations, and thus
                                        detect defects very quickly while fixing them is still cheap.

                                        [ much deletia ]

                                        > Given that Ron has already agreed that the curve is exponential, can we stop
                                        > trying to pretend it isn't, and get on with the "understanding" part?

                                        Allow me to graciously suggest that we first recognize that "cost of
                                        change" and "cost of fixing defects" are different beasts. I'm not at
                                        all sure from the conversation that everybody's talking about the same
                                        curve ...

                                        --
                                        Edmund Schweppe -- schweppe@... -- http://schweppe.home.tiac.net
                                        The opinions expressed herein are at best coincidentally related to
                                        those of any past, present or future employer.
                                      • Ron Jeffries
                                        ... Yes. However, the statement that XP (or agile methods for that matter) flatten the exponential cost curve is not a mathematical statement. It is a
                                        Message 19 of 30 , Sep 30, 2003
                                        • 0 Attachment
                                          On Tuesday, September 30, 2003, at 11:52:41 AM, acockburn@... wrote:

                                          > In a message dated 9/30/2003 9:47:00 AM Mountain Daylight Time,
                                          > ronjeffries@... writes:
                                          > <<Paraphrasing from a Wendy's commercial that I like:
                                          > Your concept of Right or Wrong in human discourse: very narrow.>>
                                          > For better or worse (both, actually) mathematics doesn't count as "human"
                                          > discourse in that sense :-)

                                          Yes. However, the statement that XP (or agile methods for that matter)
                                          "flatten the exponential cost curve" is not a mathematical statement. It is
                                          a statement of a kind of higher or alternate truth regarding the impact of
                                          short iterations and rapid feedback.

                                          In short, it's a metaphor. Doing math on a metaphor is blowing air up one's
                                          own skirt. QED.

                                          Ron Jeffries
                                          www.XProgramming.com
                                          You have to either laugh or cry. -- Bill Rogers
                                        • Alleman, Glen B.
                                          Thanks for the insight. I ve always had serious heartburn about the flattening statement. And just blown it off as something that can t get rationalized in
                                          Message 20 of 30 , Sep 30, 2003
                                          • 0 Attachment
                                            Thanks for the insight. I've always had serious heartburn about the
                                            "flattening" statement. And just blown it off as something that can't
                                            get rationalized in my math/physics pea-brain.

                                            From a metaphor view it now makes sense.

                                            Glen B. Alleman
                                            CH2M HILL
                                            Rocky Flats Environmental Technology Site
                                            glen.alleman@...
                                            glen.alleman@...

                                            >-----Original Message-----
                                            >From: Ron Jeffries [mailto:ronjeffries@...]
                                            >Sent: Tuesday, September 30, 2003 11:38 AM
                                            >To: scrumdevelopment@yahoogroups.com
                                            >Subject: Re: [scrumdevelopment] Re: flattened cost of change curve
                                            >
                                            >On Tuesday, September 30, 2003, at 11:52:41 AM, acockburn@...
                                            wrote:
                                            >
                                            >> In a message dated 9/30/2003 9:47:00 AM Mountain Daylight Time,
                                            >> ronjeffries@... writes:
                                            >> <<Paraphrasing from a Wendy's commercial that I like:
                                            >> Your concept of Right or Wrong in human discourse: very narrow.>>
                                            >> For better or worse (both, actually) mathematics doesn't count as
                                            "human"
                                            >> discourse in that sense :-)
                                            >
                                            >Yes. However, the statement that XP (or agile methods for that matter)
                                            >"flatten the exponential cost curve" is not a mathematical statement.
                                            It is
                                            >a statement of a kind of higher or alternate truth regarding the impact
                                            of
                                            >short iterations and rapid feedback.
                                            >
                                            >In short, it's a metaphor. Doing math on a metaphor is blowing air up
                                            one's
                                            >own skirt. QED.
                                            >
                                            >Ron Jeffries
                                            >www.XProgramming.com
                                            >You have to either laugh or cry. -- Bill Rogers
                                            >
                                            >
                                            >------------------------ Yahoo! Groups Sponsor
                                            ---------------------~-->
                                            >KnowledgeStorm has over 22,000 B2B technology solutions. The most
                                            >comprehensive IT buyers' information available. Research, compare,
                                            decide.
                                            >E-Commerce | Application Dev | Accounting-Finance | Healthcare |
                                            Project
                                            >Mgt | Sales-Marketing | More
                                            >http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/9EfwlB/TM
                                            >---------------------------------------------------------------------~-
                                            >
                                            >
                                            >To Post a message, send it to: scrumdevelopment@...
                                            >To Unsubscribe, send a blank message to: scrumdevelopment-
                                            >unsubscribe@...
                                            >
                                            >Your use of Yahoo! Groups is subject to
                                            http://docs.yahoo.com/info/terms/
                                            >
                                          • Ilja Preuss
                                            ... Ditto... ... I am not at all sure that Barry made this distinction. Actually I d think that having to make a change invariably had to be a reaction to a
                                            Message 21 of 30 , Oct 1, 2003
                                            • 0 Attachment
                                              Edmund Schweppe wrote:
                                              > I haven't got Kent's White Book in front of me, and I don't have
                                              > Barry's paper anywhere, so I'm speaking from oft-fragile memory here,
                                              > but ...

                                              Ditto...

                                              > AFAIK, Barry Boehm's research showed the cost of *fixing defects* grew
                                              > exponentially.
                                              >
                                              > AFAIK, Kent Beck's assertion is that the cost of *making changes*
                                              > under XP does not grow exponentially.
                                              >
                                              > AFAIK, *fixing defects* != *making changes*. More specifically, there
                                              > exist changes to software that do not involve fixing existing defects.
                                              > (Obviously, fixing a defect involves making a change. Somewhere.)

                                              I am not at all sure that Barry made this distinction. Actually I'd
                                              think that having to "make a change" invariably had to be a reaction to
                                              a "defect in the requirements" for him. At least that's the only way I
                                              can imagine introducing a "bug" in the "analysis phase" at all...

                                              Take care, Ilja
                                            • Edmund Schweppe
                                              ... I don t think he did either - and that s where I suspect a big part of the problem lies. I keep on seeing references to cost-of-*change* curves which talk
                                              Message 22 of 30 , Oct 2, 2003
                                              • 0 Attachment
                                                Ilja Preuss wrote:
                                                > Edmund Schweppe wrote:
                                                >
                                                >>I haven't got Kent's White Book in front of me, and I don't have
                                                >>Barry's paper anywhere, so I'm speaking from oft-fragile memory here,
                                                >>but ...
                                                >
                                                >
                                                > Ditto...
                                                >
                                                >
                                                >>AFAIK, Barry Boehm's research showed the cost of *fixing defects* grew
                                                >>exponentially.
                                                >>
                                                >>AFAIK, Kent Beck's assertion is that the cost of *making changes*
                                                >>under XP does not grow exponentially.
                                                >>
                                                >>AFAIK, *fixing defects* != *making changes*. More specifically, there
                                                >>exist changes to software that do not involve fixing existing defects.
                                                >>(Obviously, fixing a defect involves making a change. Somewhere.)
                                                > I am not at all sure that Barry made this distinction.

                                                I don't think he did either - and that's where I suspect a big part of
                                                the problem lies. I keep on seeing references to cost-of-*change* curves
                                                which talk about how expensive fixing *mistakes* are.

                                                > Actually I'd
                                                > think that having to "make a change" invariably had to be a reaction to
                                                > a "defect in the requirements" for him. At least that's the only way I
                                                > can imagine introducing a "bug" in the "analysis phase" at all...

                                                I can imagine two scenarios:
                                                1) When I'm initially gathering requirements from you, you tell me that
                                                the app needs to display information in both English and German. I
                                                misunderstand you (or lose my notes) and deliver an English-only app.
                                                That would be a "requirements bug".
                                                2) When I'm initially gathering requirements from you, you tell me that
                                                the app needs to display information in English. I deliver an
                                                English-only app; then you tell me that you also need the app to display
                                                information in German. (For instance, you might have been building a
                                                payroll system for an American automotive manufacturer who subsequently
                                                was purchased by a German one :-) That, to me, is a change that's not
                                                due to a bug in the requirements; instead, it's due to a change in the
                                                environment.


                                                >
                                                > Take care, Ilja
                                                >
                                                >
                                                >
                                                > To Post a message, send it to: scrumdevelopment@...
                                                > To Unsubscribe, send a blank message to: scrumdevelopment-unsubscribe@...
                                                >
                                                > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                                                >
                                                >
                                                >


                                                --
                                                Edmund Schweppe -- schweppe@... -- http://schweppe.home.tiac.net
                                                The opinions expressed herein are at best coincidentally related to
                                                those of any past, present or future employer.
                                              • Ilja Preuss
                                                ... Yep. ... Yes - though I think in a waterfallish environment it could be seen as a bug, too. You simply didn t do enough analysis upfront - else you should
                                                Message 23 of 30 , Oct 6, 2003
                                                • 0 Attachment
                                                  Edmund Schweppe wrote:
                                                  > Ilja Preuss wrote:
                                                  > > Actually I'd
                                                  >> think that having to "make a change" invariably had to be a reaction
                                                  >> to a "defect in the requirements" for him. At least that's the only
                                                  >> way I can imagine introducing a "bug" in the "analysis phase" at
                                                  >> all...
                                                  >
                                                  > I can imagine two scenarios:
                                                  > 1) When I'm initially gathering requirements from you, you tell me
                                                  > that the app needs to display information in both English and German.
                                                  > I misunderstand you (or lose my notes) and deliver an English-only
                                                  > app. That would be a "requirements bug".

                                                  Yep.

                                                  > 2) When I'm initially gathering requirements from you, you tell me
                                                  > that the app needs to display information in English. I deliver an
                                                  > English-only app; then you tell me that you also need the app to
                                                  > display information in German. (For instance, you might have been
                                                  > building a payroll system for an American automotive manufacturer who
                                                  > subsequently was purchased by a German one :-) That, to me, is a
                                                  > change that's not due to a bug in the requirements; instead, it's due
                                                  > to a change in the environment.

                                                  Yes - though I think in a waterfallish environment it could be seen as a
                                                  bug, too. You simply didn't do enough analysis upfront - else you should
                                                  have known that you could need a different language version in the
                                                  future. Because you didn't, you have to restart the whole development
                                                  cycle.

                                                  To an agile team, of course, *both* scenarios just result in another
                                                  feature to be added incrementally. That's why I think that "requirement
                                                  bugs" aren't that important in agile development.

                                                  Regards, Ilja
                                                Your message has been successfully submitted and would be delivered to recipients shortly.