Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Microsoft Research on TDD

Expand Messages
  • George Dinwiddie
    Pooya, ... If you re interested in such papers, you might find http://biblio.gdinwiddie.com/biblio/StudiesOfTestDrivenDevelopment of use. There are a few other
    Message 1 of 10 , Dec 30, 2012
    • 0 Attachment
      Pooya,

      On 12/29/12 11:48 PM, pooya shahbazian wrote:
      > hi jeff
      > thanks for introducing the paper.

      If you're interested in such papers, you might find
      http://biblio.gdinwiddie.com/biblio/StudiesOfTestDrivenDevelopment of
      use. There are a few other papers along this line.

      - George

      --
      ----------------------------------------------------------------------
      * George Dinwiddie * http://blog.gdinwiddie.com
      Software Development http://www.idiacomputing.com
      Consultant and Coach http://www.agilemaryland.org
      ----------------------------------------------------------------------
    • Ron Jeffries
      Hi Jeff, Good stuff. ... What is unfortunate in these studies, and in all the studies I ve seen, is that they seem to be estimating the time to produce
      Message 2 of 10 , Dec 30, 2012
      • 0 Attachment
        Hi Jeff,

        Good stuff.
        On Dec 29, 2012, at 8:31 PM, JeffGrigg <jeffreytoddgrigg@...> wrote:

        > An important paper by Microsoft research was referenced indirectly in on e of the current threads.
        >
        > It is "Realizing quality improvement through test driven
        > development: results and experiences of four industrial
        > teams" by Nachiappan Nagappan, E. Michael Maximilien,
        > Thirumalesh Bhat, and Laurie Williams.
        >
        > reference:
        > http://link.springer.com/article/10.1007%2Fs10664-008-9062-z
        >
        > full paper:
        > http://research.microsoft.com/en-us/groups/ese/nagappan_tdd.pdf
        >
        > It reports 40% to 90% defect reduction and 15% to 35% increase in cost.

        > HOWEVER, before everyone runs off and plans projects based on those numbers, I'd like to point out that while the defect reduction numbers were measurements, the "cost increases" were "shoot from the hip estimates" by the bosses.

        What is unfortunate in these studies, and in all the studies I've seen, is that they seem to be estimating the time to produce bug-free code on the TDD side, and buggy code on the non-TDD side, and do not add in the time to fix the bugs that need to be fixed. So not only are those numbers pulled from the hip or lower, they're probably not comparable even if they were accurate.

        But still good stuff!

        Ron Jeffries
        www.XProgramming.com
        Wisdom begins when we learn the difference between "that makes no sense" and "I don't understand". -- Mary Doria Russell



        [Non-text portions of this message have been removed]
      • M. Manca
        Il 30/12/2012 02:31, JeffGrigg ha scritto: Hi Jeff, I have similar results about quality but about cost I have a reduction between 25% to 45% compared with
        Message 3 of 10 , Dec 30, 2012
        • 0 Attachment
          Il 30/12/2012 02:31, JeffGrigg ha scritto:
          Hi Jeff,
          I have similar results about quality but about cost I have a reduction
          between 25% to 45% compared with similar embedded projects.
          >
          >
          > An important paper by Microsoft research was referenced indirectly in
          > on e of the current threads.
          >
          > It is "Realizing quality improvement through test driven
          > development: results and experiences of four industrial
          > teams" by Nachiappan Nagappan, E. Michael Maximilien,
          > Thirumalesh Bhat, and Laurie Williams.
          >
          > reference:
          > http://link.springer.com/article/10.1007%2Fs10664-008-9062-z
          >
          > full paper:
          > http://research.microsoft.com/en-us/groups/ese/nagappan_tdd.pdf
          >
          > It reports 40% to 90% defect reduction and 15% to 35% increase in cost.
          >
          > HOWEVER, before everyone runs off and plans projects based on those
          > numbers, I'd like to point out that while the defect reduction numbers
          > were measurements, the "cost increases" were "shoot from the hip
          > estimates" by the bosses.
          >
          > The paper says...
          > "Another interesting observation from the outcome measures in Table 3
          > is the increase in time to develop the features attributed to the
          > usage of the TDD practice, as subjectively estimated by management.
          > The increase in development time ranges from 15% to 35%. From an
          > efficacy perspective this increase in development time is offset by
          > the by the reduced maintenance costs due to the improvement in quality
          > (Erdogmus and Williams 2003), an observation that was backed up the
          > product teams at Microsoft and IBM."
          >
          > I'm emphasizing the "as subjectively estimated by management" part.
          >
          >



          [Non-text portions of this message have been removed]
        • George Dinwiddie
          Ron, ... Yes, I can write code really fast if it doesn t have to work. - George -- ... * George Dinwiddie * http://blog.gdinwiddie.com
          Message 4 of 10 , Dec 30, 2012
          • 0 Attachment
            Ron,

            On 12/30/12 6:10 AM, Ron Jeffries wrote:
            > Hi Jeff,
            >
            > Good stuff.
            > On Dec 29, 2012, at 8:31 PM, JeffGrigg <jeffreytoddgrigg@...> wrote:
            >
            >> An important paper by Microsoft research was referenced indirectly in on e of the current threads.
            >>
            >> It is "Realizing quality improvement through test driven
            >> development: results and experiences of four industrial
            >> teams" by Nachiappan Nagappan, E. Michael Maximilien,
            >> Thirumalesh Bhat, and Laurie Williams.
            >>
            >> reference:
            >> http://link.springer.com/article/10.1007%2Fs10664-008-9062-z
            >>
            >> full paper:
            >> http://research.microsoft.com/en-us/groups/ese/nagappan_tdd.pdf
            >>
            >> It reports 40% to 90% defect reduction and 15% to 35% increase in cost.
            >
            >> HOWEVER, before everyone runs off and plans projects based on those
            >> numbers, I'd like to point out that while the defect reduction
            >> numbers were measurements, the "cost increases" were "shoot from
            >> the hip estimates" by the bosses.
            >
            > What is unfortunate in these studies, and in all the studies I've
            > seen, is that they seem to be estimating the time to produce bug-free
            > code on the TDD side, and buggy code on the non-TDD side, and do not
            > add in the time to fix the bugs that need to be fixed. So not only
            > are those numbers pulled from the hip or lower, they're probably not
            > comparable even if they were accurate.

            Yes, I can write code really fast if it doesn't have to work.

            - George

            --
            ----------------------------------------------------------------------
            * George Dinwiddie * http://blog.gdinwiddie.com
            Software Development http://www.idiacomputing.com
            Consultant and Coach http://www.agilemaryland.org
            ----------------------------------------------------------------------
          • pooya shahbazian
            hi thanks, it s so useful  Pooya Shahbazian http://www.pooyablog.blogfa.com/ ________________________________ From: George Dinwiddie
            Message 5 of 10 , Dec 30, 2012
            • 0 Attachment
              hi
              thanks, it's so useful 

              Pooya Shahbazian
              http://www.pooyablog.blogfa.com/



              ________________________________
              From: George Dinwiddie <lists@...>
              To: extremeprogramming@yahoogroups.com
              Sent: Sunday, December 30, 2012 2:03 PM
              Subject: Re: [XP] Microsoft Research on TDD


               

              Pooya,

              On 12/29/12 11:48 PM, pooya shahbazian wrote:
              > hi jeff
              > thanks for introducing the paper.

              If you're interested in such papers, you might find
              http://biblio.gdinwiddie.com/biblio/StudiesOfTestDrivenDevelopment of
              use. There are a few other papers along this line.

              - George

              --
              ----------------------------------------------------------
              * George Dinwiddie * http://blog.gdinwiddie.com
              Software Development http://www.idiacomputing.com
              Consultant and Coach http://www.agilemaryland.org
              ----------------------------------------------------------




              [Non-text portions of this message have been removed]
            • JeffGrigg
              ... Yes; some studies seem to ignore the fact that companies typically spend 90% to 99% or more of their software development budgets on maintenance. So the
              Message 6 of 10 , Dec 30, 2012
              • 0 Attachment
                --- "M. Manca" <m.manca@...> wrote:
                > Hi Jeff,
                > I have similar results about quality but about cost I
                > have a reduction between 25% to 45% compared with
                > similar embedded projects.

                Yes; some studies seem to ignore the fact that companies typically spend 90% to 99% or more of their software development budgets on maintenance. So the decreased maintenance costs of having fewer bugs, and more readable and maintainable code, and in-place test suites to prevent regression, are often overlooked.

                Even during the initial development phase, I've seen numerous benefits to the TDD approach. A friend of mine in the early 2000's studied a number of projects done in the same company with similar teams, similar management, the same tools, etc. -- giving him a good comparison of XP to their more conventional approaches. His numbers showed that not using XP cost the company two to four times as much -- just during the initial development project.
              Your message has been successfully submitted and would be delivered to recipients shortly.