Loading ...
Sorry, an error occurred while loading the content.

Balance between writing unit tests and shipping code

Expand Messages
  • John Cornell
    As part of my role as an Agile coach in my corporation, I am often asked by Scrum team members how much time they should spend writing unit tests compared to
    Message 1 of 8 , Jan 31, 2007
    • 0 Attachment

      As part of my role as an Agile coach in my corporation, I am often asked by Scrum team members how much time they should spend writing unit tests compared to writing code to implement the user story.  Just to be clear, I am asking about unit tests written using frameworks such as JUnit and NUnit and not asking about automated testing using tools such as Mercury, TestComplete, etc.  Does anyone have hard metrics on this balance or is it more of a story by story decision?  How have others addressed the concern raised by software engineers that they are spending too much time on writing unit tests?  Feel free to just point me to an article if this is an obvious question that has already been discussed ad nauseam.  Thanks

       

      John Cornell

      Kofax Image Products

      Software Development Manager

      http://www.kofax.com/

       

    • Keith Ray
      I would say that if they are using about 40% - 60% of their time writing automated unit tests versus writing production code, they are probably in a good
      Message 2 of 8 , Jan 31, 2007
      • 0 Attachment
        I would say that if they are using about 40% - 60% of their time
        writing automated unit tests versus writing production code, they are
        probably in a good balance. Add in time for writing automated
        acceptance tests and the percentage would be even higher.

        That might be enough for some developers to cry foul - saying that all
        that testing is wasting their time; but without the automated tests,
        the team is going to retesting every sprint (and every day) manually.

        The pain of unit testing can be alleviated by learning Test Driven
        Development (TDD). And with TDD, the time spent debugging will drop
        to close to zero percent.

        Ask your developers what percentage of their time do they spend in the
        debugger and/or manually testing their code. THAT time is often one of
        the biggest wastes.

        On 1/31/07, John Cornell <john_cornell@...> wrote:
        >
        >
        >
        >
        > As part of my role as an Agile coach in my corporation, I am often asked by
        > Scrum team members how much time they should spend writing unit tests
        > compared to writing code to implement the user story. Just to be clear, I
        > am asking about unit tests written using frameworks such as JUnit and NUnit
        > and not asking about automated testing using tools such as Mercury,
        > TestComplete, etc. Does anyone have hard metrics on this balance or is it
        > more of a story by story decision? How have others addressed the concern
        > raised by software engineers that they are spending too much time on writing
        > unit tests? Feel free to just point me to an article if this is an obvious
        > question that has already been discussed ad nauseam. Thanks
        >
        >
        >
        > John Cornell
        >
        > Kofax Image Products
        >
        > Software Development Manager
        >
        > http://www.kofax.com/
        >
        >


        --
        ----
        I n d u s t r i a l L o g i c , I n c .
        C. Keith Ray
        http://industriallogic.com
        http://industrialxp.org
        866-540-8336 (toll free)
        510-540-8336 (phone)
        Berkeley, California
      • Gupta, Samit (ELS-CAM)
        I would agree with 40%-60% approach as the first pass but if you are focusing on proper test coverage the developers may have to re-visit the test cases to
        Message 3 of 8 , Jan 31, 2007
        • 0 Attachment

          I would agree with 40%-60% approach as the first pass but if you are focusing

          on proper test coverage the developers may have to re-visit the test cases to

          make sure the test coverage is appropriate.

           

          My team makes sure that story estimates consider time to write unit tests to

          at least cover all acceptance criteria for the given story.

           

          ________________________________________________________

          Samit Gupta :: Development Manager :: Agile Projects :: ELSEVIER ::

          Camden Office :: T: +44 (0)207 424 4416  :: F: +44 (0)207 424 4544 ::
          IT eDevelopment :: 32 Jamestown Road :: London :: NW1 7BY ::

          s.gupta@... :www.elsevier.com

          ________________________________________________________

           


          From: scrumdevelopment@yahoogroups.com [mailto:scrumdevelopment@yahoogroups.com] On Behalf Of Keith Ray
          Sent: 31 January 2007 21:47
          To: scrumdevelopment@yahoogroups.com
          Subject: Re: [scrumdevelopment] Balance between writing unit tests and shipping code

           

          I would say that if they are using about 40% - 60% of their time
          writing automated unit tests versus writing production code, they are
          probably in a good balance. Add in time for writing automated
          acceptance tests and the percentage would be even higher.

          That might be enough for some developers to cry foul - saying that all
          that testing is wasting their time; but without the automated tests,
          the team is going to retesting every sprint (and every day) manually.

          The pain of unit testing can be alleviated by learning Test Driven
          Development (TDD). And with TDD, the time spent debugging will drop
          to close to zero percent.

          Ask your developers what percentage of their time do they spend in the
          debugger and/or manually testing their code. THAT time is often one of
          the biggest wastes.

          On 1/31/07, John Cornell <john_cornell@ kofax.com> wrote:

          >
          >
          >
          >
          > As part of my role as an Agile coach in my corporation, I am often asked
          by
          > Scrum team members how much time they should spend writing unit tests
          > compared to writing code to implement the user story. Just to be clear, I
          > am asking about unit tests written using frameworks such as JUnit and
          NUnit
          > and not asking about automated testing using tools such as Mercury,
          > TestComplete, etc. Does anyone have hard metrics on this balance or is it
          > more of a story by story decision? How have others addressed the concern
          > raised by software engineers that they are spending too much time on
          writing
          > unit tests? Feel free to just point me to an article if this is an obvious
          > question that has already been discussed ad nauseam. Thanks
          >
          >
          >
          > John Cornell
          >
          > Kofax Image Products
          >
          > Software Development Manager
          >
          > http://www.kofax. com/

        • Ron Jeffries
          Hello, John. On Wednesday, January 31, 2007, at 1:23:01 PM, you ... If you do test-driven development, you get the right answer automatically, because you
          Message 4 of 8 , Jan 31, 2007
          • 0 Attachment
            Hello, John. On Wednesday, January 31, 2007, at 1:23:01 PM, you
            wrote:

            > As part of my role as an Agile coach in my corporation, I am often asked
            > by Scrum team members how much time they should spend writing unit tests
            > compared to writing code to implement the user story. Just to be clear, I
            > am asking about unit tests written using frameworks such as JUnit and
            > NUnit and not asking about automated testing using tools such as Mercury,
            > TestComplete, etc. Does anyone have hard metrics on this balance or is it
            > more of a story by story decision? How have others addressed the concern
            > raised by software engineers that they are spending too much time on
            > writing unit tests? Feel free to just point me to an article if this is
            > an obvious question that has already been discussed ad nauseam. Thanks

            If you do test-driven development, you get the right answer
            automatically, because you never write any code except after you
            write a test.

            If you write test until you are confident that the program works,
            you get the right answer because you won't bother to write tests for
            a program that you're confident about. (And then, when bug reports
            disabuse you of your overconfidence, you'll dial up the testing a
            bit.)

            It's not about hours. It's about enough.

            Ron Jeffries
            www.XProgramming.com
            For me, XP ain't out there, it's in here. -- Bill Caputo
          • Nicholas Cancelliere
            I do not think there is a one-size-fits-all answer to this. If you are using JUnit or NUnit you should be writing those tests before you develop. You can use
            Message 5 of 8 , Jan 31, 2007
            • 0 Attachment
              I do not think there is a one-size-fits-all answer to this.  If you are using JUnit or NUnit you should be writing those tests before you develop.  You can use tools like Emma to produce coverage reports.  Depending on how critical a component is you might require coverage to be higher than something that's less important.  For example - a billing package might be required to be 95%+ covered while a convenience package (like a subsystem that emails "my favorite links" to your friends) might be okay at 85% or more.  It all depends on how much risk you're willing to take and how much uncertainty you can live with.
               
              In the end lower coverage will result in higher defects and more bugs.  You can send the time up-front to write good tests or wait until bugs are found, customers lose confidence and spend the time there.  Consider too, it's easier to plan/understand writing tests as (or before) you develop than to anticipate a defect later and how it might impact other work in progress.  Defects are bad because they add uncertainty and drag to your project.  Having a high code coverage also allows you the freed to refactor without wondering "did I break anything by what I just changed?"
               
              -Nick
               
               
              ----- Original Message -----
              Sent: Wednesday, January 31, 2007 3:23 PM
              Subject: [scrumdevelopment] Balance between writing unit tests and shipping code

              As part of my role as an Agile coach in my corporation, I am often asked by Scrum team members how much time they should spend writing unit tests compared to writing code to implement the user story.  Just to be clear, I am asking about unit tests written using frameworks such as JUnit and NUnit and not asking about automated testing using tools such as Mercury, TestComplete, etc.  Does anyone have hard metrics on this balance or is it more of a story by story decision?  How have others addressed the concern raised by software engineers that they are spending too much time on writing unit tests?  Feel free to just point me to an article if this is an obvious question that has already been discussed ad nauseam.  Thanks

              John Cornell

              Kofax Image Products

              Software Development Manager

              http://www.kofax. com/

            • Wolfgang Schulze Zachau
              (replying to Keith and John) considering that in all but the most trivial cases you will need at least 2 unit tests per method (and in some cases up to several
              Message 6 of 8 , Feb 1, 2007
              • 0 Attachment
                (replying to Keith and John)
                 
                considering that in all but the most trivial cases you will need at least 2 unit tests per method (and in some cases up to several dozen to cover all possible combinations of inputs and all possible results), it would be realistic to say that the size of test code will always dwarf the size of the application code (particularly after serious refactoring). In consequence there is nothing wrong with spending lots of time on the test code. As others have pointed out, if your tests have good coverage and fine granularity, developers will reduce spending time in the debugger considerably. In return you will get  high quality code.
                 
                In fact, the quality of the code will follow exactly the quality of the requirements provided by the product owner. The more precisely and comprehensively the PO can state his acceptance criteria, the more better your test coverage will be and in consequence the better the code will match the business requirements.
                 
                I suggest you and the team try TDD with a relatively simple (yet real !!) part of the project. Follow the book by the letter and see the results. It is truly amazing, but be prepared to change paradigms on the way.
                 

                Regards,

                Wolfgang

              • Andy
                ... Took the words right out of my mouth. You don t introduce new functionality, or fix broken functionality, unless there s a test to cover the issue you re
                Message 7 of 8 , Feb 1, 2007
                • 0 Attachment
                  --- In scrumdevelopment@yahoogroups.com, Ron Jeffries
                  <ronjeffries@...> wrote:
                  >
                  > It's not about hours. It's about enough.
                  >

                  Took the words right out of my mouth. You don't introduce new
                  functionality, or fix broken functionality, unless there's a test to
                  cover the issue you're trying to address.

                  If you haven't written that test, you haven't spent enough time on it.

                  This assumes you're doing TDD or BDD of course. After 15 months of
                  doing it, I can't imagine coding any other way. Coding anything other
                  than spike solutions without TDD has become almost unthinkable.

                  Andy.
                • Anoshlal Sulgaonkar
                  We write the acceptance test using a testing tool known as STIQ (Story test IQ). We consider writing tests as a part of coading. We write the tests first and
                  Message 8 of 8 , Feb 1, 2007
                  • 0 Attachment
                    We write the acceptance test using a testing tool known as STIQ (Story test IQ). We consider writing tests as a part of coading. We write the tests first and then code to pass the tests. This makes our lives easy.


                    Cheap Talk? Check out Yahoo! Messenger's low PC-to-Phone call rates.
                  Your message has been successfully submitted and would be delivered to recipients shortly.