Loading ...
Sorry, an error occurred while loading the content.

Diminishing ROI with NO Automation in Agile development

Expand Messages
  • Hiren
    My interaction with various clients that are in process of transitioning to Agile often ask this question - Is it ok to skip automated Unit and Acceptance
    Message 1 of 7 , Oct 4, 2010
    • 0 Attachment
      My interaction with various clients that are in process of transitioning to Agile often ask this question - Is it ok to skip automated Unit and Acceptance testing and only perform manual testing? The reservation normally arises due to old school of thought where manual testing was tolerated and where CLIs and API needed for testing are normally an after thought. Sometimes it is also because QA engineers are manual testers and have no needed programming skills. And at times people tell me Automation takes time off from active feature development and is an overhead.

      No matter what is the reasoning, I always preach to these teams, "Automation is a rule and not an exception" and I try to justify my statement using the example below. 

      For complete blog please follow the this link 

      Please share your comments.

      Thank you.
      -Hiren
      Blog:  PracticeAgile 


    • Roy Morien
      Hi Hiren, You have clearly stated the problem already ... the old school of thought ... testing normally an afterthought ... QA engineers ... QA
      Message 2 of 7 , Oct 4, 2010
      • 0 Attachment
        Hi Hiren,
         
        You have clearly stated the problem already ... 'the old school of thought' ... 'testing normally an afterthought' ... 'QA engineers' ... 'QA engineers are manual testers'.
         
        So introducing automated testing would require a siginifcant change in attitude and practice, AND the introduction of a new set of tools, with the need for training and education that goes with it.
         
        So ... I would say that automated testing is highly desireable at least, but you will need to implement a significant program of adoption and acceptance (or is that acceptance and adoption?). This will not be easy.
         
        But I am merely reflecting what you have already indicated. If you have such pushback from these folk, then maybe you are fighting a battle that you might not be ab;e to win, at least in the short term. Change always has its opponents.
         
        Regards,
        Roy Morien
         

        To: scrumdevelopment@yahoogroups.com
        From: hirendoshi73@...
        Date: Mon, 4 Oct 2010 07:26:32 +0000
        Subject: [scrumdevelopment] Diminishing ROI with NO Automation in Agile development

         
        My interaction with various clients that are in process of transitioning to Agile often ask this question - Is it ok to skip automated Unit and Acceptance testing and only perform manual testing? The reservation normally arises due to old school of thought where manual testing was tolerated and where CLIs and API needed for testing are normally an after thought. Sometimes it is also because QA engineers are manual testers and have no needed programming skills. And at times people tell me Automation takes time off from active feature development and is an overhead.

        No matter what is the reasoning, I always preach to these teams, "Automation is a rule and not an exception" and I try to justify my statement using the example below. 

        For complete blog please follow the this link 

        Please share your comments.

        Thank you.
        -Hiren
        Blog:  PracticeAgile 



      • Ron Jeffries
        Hello, Hiren. On Monday, October 4, 2010, at 3:26:32 AM, you ... It s not a rule. Rules do not work in governing human behavior. We automate tests for these
        Message 3 of 7 , Oct 4, 2010
        • 0 Attachment
          Hello, Hiren. On Monday, October 4, 2010, at 3:26:32 AM, you
          wrote:

          > No matter what is the reasoning, I always preach to these teams,
          > "Automation is a rule and not an exception"

          It's not a rule. Rules do not work in governing human behavior.

          We automate tests for these reasons:

          1. Test automation provides faster feedback. If a developer has
          an automated test for some feature, he knows whether it works or
          does not work according to the specification (namely the test).

          2. It prevents regressions. In a well-designed software system,
          changes to one module can affect many others: that's how
          modularity works. In a poorly-designed system, the impact of
          change is even less predictable. Automated tests give us a fast
          way to know whether we have broken something.

          3. It saves time. The system needs lots of testing, and repetitive
          testing. Doing it with humans slows us down.

          4. It prevents mistakes. If we test with humans, we will have to
          decide which tests not to do. Often, we will decide incorrectly,
          letting critical errors slip in. Often, humans even make mistakes
          in the tests that they perform, again letting errors slip in.

          Automated testing is not a rule. It is a powerful practice that lets
          the team go faster with far fewer errors.

          This does not mean that every test should be automated. It does
          mean, however, that if a team is new to automated testing, almost
          every test they are not doing probably should be automated, because
          they will be so far behind that they are likely not doing very much
          really good testing.

          Ron Jeffries
          www.XProgramming.com
          www.xprogramming.com/blog
          The greatest mistake we make is living in constant fear that we will make one.
          -- John Maxwell
        • Adam Sroka
          Hi Ron: Great Post... ... I had to read this sentence several times to make sure I parsed it correctly. I ll say what I think you mean, and then you can
          Message 4 of 7 , Oct 4, 2010
          • 0 Attachment
            Hi Ron:

            Great Post...

            On Mon, Oct 4, 2010 at 3:25 AM, Ron Jeffries <ronjeffries@...> wrote:
            >
            > It does
            > mean, however, that if a team is new to automated testing, almost
            > every test they are not doing probably should be automated, because
            > they will be so far behind that they are likely not doing very much
            > really good testing.
            >

            I had to read this sentence several times to make sure I parsed it
            correctly. I'll say what I think you mean, and then you can correct me
            ;-)

            I think you are saying that any test that is not being done manually
            should be automated, because otherwise there will be too many tests
            that are not getting done. Is that right?

            ...

            I would also like to point out the Pragmatic Programmers' article on
            Ubiquitous Automation
            (http://media.pragprog.com/articles/jan_02_auto.pdf) which makes a lot
            of the same points, expands upon them, and generalizes them to
            /everything/ we do, not just testing.
          • Ron Jeffries
            ... Let me try. If your team is new to automated testing, it is likely that most every test you are now doing should be automated. The reason is that if you
            Message 5 of 7 , Oct 4, 2010
            • 0 Attachment
              Hello, Adam. On Monday, October 4, 2010, at 2:46:42 PM, you wrote:

              >> It does
              >> mean, however, that if a team is new to automated testing, almost
              >> every test they are not doing probably should be automated, because
              >> they will be so far behind that they are likely not doing very much
              >> really good testing.
              >>

              > I had to read this sentence several times to make sure I parsed it
              > correctly. I'll say what I think you mean, and then you can correct me
              > ;-)

              > I think you are saying that any test that is not being done manually
              > should be automated, because otherwise there will be too many tests
              > that are not getting done. Is that right?

              Let me try.

              If your team is new to automated testing, it is likely that most
              every test you are now doing should be automated. The reason is that
              if you are testing manually, you are probably doing very few tests
              that only a human can do.

              So my advice would be to err on the side of over-automating for a
              while.

              Ron Jeffries
              www.XProgramming.com
              www.xprogramming.com/blog
              [S]oftware engineering ... "How to program if you cannot." -- Edsger Dijkstra
            • Don MacIntyre
              I describe the need for the adoption of automated testing and other XP practices to new Scrum teams something like this... If you don t adopt the practices
              Message 6 of 7 , Oct 5, 2010
              • 0 Attachment
                I describe the need for the adoption of automated testing and other XP practices to new Scrum teams something like this...

                If you don't adopt the practices that are best suited for iterative development, you may experience an initial burst in productivity, but you will likely soon discover that you are just building crap faster than you were before.

                I feel that continuous integration and automated testing are necessary for iterative development.

                You can do typical unit testing and have your CI system kick off other automated suites (we currently use selenium) but the biggest bang for the buck is Test Driven Development.

                My current teams have been slow to totally embrace TDD, so I'm making a real push now with weekly how-to sessions. One of the guys will also spend the next 3 days at Ron and Chet's Agile Skills class. Hopefully that will be enough to drive it home. ;)

                -don
                http://www.scrumalliance.org/profiles/26043-don-macintyre

                --- In scrumdevelopment@yahoogroups.com, Ron Jeffries <ronjeffries@...> wrote:
                >
                > Hello, Adam. On Monday, October 4, 2010, at 2:46:42 PM, you wrote:
                >
                > >> It does
                > >> mean, however, that if a team is new to automated testing, almost
                > >> every test they are not doing probably should be automated, because
                > >> they will be so far behind that they are likely not doing very much
                > >> really good testing.
                > >>
                >
                > > I had to read this sentence several times to make sure I parsed it
                > > correctly. I'll say what I think you mean, and then you can correct me
                > > ;-)
                >
                > > I think you are saying that any test that is not being done manually
                > > should be automated, because otherwise there will be too many tests
                > > that are not getting done. Is that right?
                >
                > Let me try.
                >
                > If your team is new to automated testing, it is likely that most
                > every test you are now doing should be automated. The reason is that
                > if you are testing manually, you are probably doing very few tests
                > that only a human can do.
                >
                > So my advice would be to err on the side of over-automating for a
                > while.
                >
                > Ron Jeffries
                > www.XProgramming.com
                > www.xprogramming.com/blog
                > [S]oftware engineering ... "How to program if you cannot." -- Edsger Dijkstra
                >
              • woynam
                Perhaps I work in a different type of IT shop than the typical developer, but I m really astonished that in 2010 someone can suggest that we can live without
                Message 7 of 7 , Oct 5, 2010
                • 0 Attachment
                  Perhaps I work in a different type of IT shop than the typical developer, but I'm really astonished that in 2010 someone can suggest that we can live without automated tests. (The comment is not directed at you, Don.)

                  Our system consists of over 2 million lines of code. A single test plan for a single complex feature takes ~70 pages to print, given that there are hundreds of variations.

                  If you printed the entire set of tests (single sided), it would be over 6 feet tall. I know, we did it once for fun, and to provide a important visual clue. :-)

                  Before we automated the vast majority of the tests, it would take over a month to manually execute the test suite.

                  Now, how on earth can someone suggest that they're doing agile development, delivering working software every few weeks, when the execution of manual tests alone would account for weeks of time?

                  On top of that, the vast majority of (regression) tests do not fail when they're run each the end of each iteration. What a colossal waste of time it would be to have your talented team poking away at the system, when they could be adding value elsewhere.

                  Mark


                  --- In scrumdevelopment@yahoogroups.com, "Don MacIntyre" <don@...> wrote:
                  >
                  > I describe the need for the adoption of automated testing and other XP practices to new Scrum teams something like this...
                  >
                  > If you don't adopt the practices that are best suited for iterative development, you may experience an initial burst in productivity, but you will likely soon discover that you are just building crap faster than you were before.
                  >
                  > I feel that continuous integration and automated testing are necessary for iterative development.
                  >
                  > You can do typical unit testing and have your CI system kick off other automated suites (we currently use selenium) but the biggest bang for the buck is Test Driven Development.
                  >
                  > My current teams have been slow to totally embrace TDD, so I'm making a real push now with weekly how-to sessions. One of the guys will also spend the next 3 days at Ron and Chet's Agile Skills class. Hopefully that will be enough to drive it home. ;)
                  >
                  > -don
                  > http://www.scrumalliance.org/profiles/26043-don-macintyre
                  >
                  > --- In scrumdevelopment@yahoogroups.com, Ron Jeffries <ronjeffries@> wrote:
                  > >
                  > > Hello, Adam. On Monday, October 4, 2010, at 2:46:42 PM, you wrote:
                  > >
                  > > >> It does
                  > > >> mean, however, that if a team is new to automated testing, almost
                  > > >> every test they are not doing probably should be automated, because
                  > > >> they will be so far behind that they are likely not doing very much
                  > > >> really good testing.
                  > > >>
                  > >
                  > > > I had to read this sentence several times to make sure I parsed it
                  > > > correctly. I'll say what I think you mean, and then you can correct me
                  > > > ;-)
                  > >
                  > > > I think you are saying that any test that is not being done manually
                  > > > should be automated, because otherwise there will be too many tests
                  > > > that are not getting done. Is that right?
                  > >
                  > > Let me try.
                  > >
                  > > If your team is new to automated testing, it is likely that most
                  > > every test you are now doing should be automated. The reason is that
                  > > if you are testing manually, you are probably doing very few tests
                  > > that only a human can do.
                  > >
                  > > So my advice would be to err on the side of over-automating for a
                  > > while.
                  > >
                  > > Ron Jeffries
                  > > www.XProgramming.com
                  > > www.xprogramming.com/blog
                  > > [S]oftware engineering ... "How to program if you cannot." -- Edsger Dijkstra
                  > >
                  >
                Your message has been successfully submitted and would be delivered to recipients shortly.