Loading ...
Sorry, an error occurred while loading the content.

RE: [scrumdevelopment] Diminishing ROI with NO Automation in Agile development

Expand Messages
  • Roy Morien
    Hi Hiren, You have clearly stated the problem already ... the old school of thought ... testing normally an afterthought ... QA engineers ... QA
    Message 1 of 7 , Oct 4, 2010
    • 0 Attachment
      Hi Hiren,
       
      You have clearly stated the problem already ... 'the old school of thought' ... 'testing normally an afterthought' ... 'QA engineers' ... 'QA engineers are manual testers'.
       
      So introducing automated testing would require a siginifcant change in attitude and practice, AND the introduction of a new set of tools, with the need for training and education that goes with it.
       
      So ... I would say that automated testing is highly desireable at least, but you will need to implement a significant program of adoption and acceptance (or is that acceptance and adoption?). This will not be easy.
       
      But I am merely reflecting what you have already indicated. If you have such pushback from these folk, then maybe you are fighting a battle that you might not be ab;e to win, at least in the short term. Change always has its opponents.
       
      Regards,
      Roy Morien
       

      To: scrumdevelopment@yahoogroups.com
      From: hirendoshi73@...
      Date: Mon, 4 Oct 2010 07:26:32 +0000
      Subject: [scrumdevelopment] Diminishing ROI with NO Automation in Agile development

       
      My interaction with various clients that are in process of transitioning to Agile often ask this question - Is it ok to skip automated Unit and Acceptance testing and only perform manual testing? The reservation normally arises due to old school of thought where manual testing was tolerated and where CLIs and API needed for testing are normally an after thought. Sometimes it is also because QA engineers are manual testers and have no needed programming skills. And at times people tell me Automation takes time off from active feature development and is an overhead.

      No matter what is the reasoning, I always preach to these teams, "Automation is a rule and not an exception" and I try to justify my statement using the example below. 

      For complete blog please follow the this link 

      Please share your comments.

      Thank you.
      -Hiren
      Blog:  PracticeAgile 



    • Ron Jeffries
      Hello, Hiren. On Monday, October 4, 2010, at 3:26:32 AM, you ... It s not a rule. Rules do not work in governing human behavior. We automate tests for these
      Message 2 of 7 , Oct 4, 2010
      • 0 Attachment
        Hello, Hiren. On Monday, October 4, 2010, at 3:26:32 AM, you
        wrote:

        > No matter what is the reasoning, I always preach to these teams,
        > "Automation is a rule and not an exception"

        It's not a rule. Rules do not work in governing human behavior.

        We automate tests for these reasons:

        1. Test automation provides faster feedback. If a developer has
        an automated test for some feature, he knows whether it works or
        does not work according to the specification (namely the test).

        2. It prevents regressions. In a well-designed software system,
        changes to one module can affect many others: that's how
        modularity works. In a poorly-designed system, the impact of
        change is even less predictable. Automated tests give us a fast
        way to know whether we have broken something.

        3. It saves time. The system needs lots of testing, and repetitive
        testing. Doing it with humans slows us down.

        4. It prevents mistakes. If we test with humans, we will have to
        decide which tests not to do. Often, we will decide incorrectly,
        letting critical errors slip in. Often, humans even make mistakes
        in the tests that they perform, again letting errors slip in.

        Automated testing is not a rule. It is a powerful practice that lets
        the team go faster with far fewer errors.

        This does not mean that every test should be automated. It does
        mean, however, that if a team is new to automated testing, almost
        every test they are not doing probably should be automated, because
        they will be so far behind that they are likely not doing very much
        really good testing.

        Ron Jeffries
        www.XProgramming.com
        www.xprogramming.com/blog
        The greatest mistake we make is living in constant fear that we will make one.
        -- John Maxwell
      • Adam Sroka
        Hi Ron: Great Post... ... I had to read this sentence several times to make sure I parsed it correctly. I ll say what I think you mean, and then you can
        Message 3 of 7 , Oct 4, 2010
        • 0 Attachment
          Hi Ron:

          Great Post...

          On Mon, Oct 4, 2010 at 3:25 AM, Ron Jeffries <ronjeffries@...> wrote:
          >
          > It does
          > mean, however, that if a team is new to automated testing, almost
          > every test they are not doing probably should be automated, because
          > they will be so far behind that they are likely not doing very much
          > really good testing.
          >

          I had to read this sentence several times to make sure I parsed it
          correctly. I'll say what I think you mean, and then you can correct me
          ;-)

          I think you are saying that any test that is not being done manually
          should be automated, because otherwise there will be too many tests
          that are not getting done. Is that right?

          ...

          I would also like to point out the Pragmatic Programmers' article on
          Ubiquitous Automation
          (http://media.pragprog.com/articles/jan_02_auto.pdf) which makes a lot
          of the same points, expands upon them, and generalizes them to
          /everything/ we do, not just testing.
        • Ron Jeffries
          ... Let me try. If your team is new to automated testing, it is likely that most every test you are now doing should be automated. The reason is that if you
          Message 4 of 7 , Oct 4, 2010
          • 0 Attachment
            Hello, Adam. On Monday, October 4, 2010, at 2:46:42 PM, you wrote:

            >> It does
            >> mean, however, that if a team is new to automated testing, almost
            >> every test they are not doing probably should be automated, because
            >> they will be so far behind that they are likely not doing very much
            >> really good testing.
            >>

            > I had to read this sentence several times to make sure I parsed it
            > correctly. I'll say what I think you mean, and then you can correct me
            > ;-)

            > I think you are saying that any test that is not being done manually
            > should be automated, because otherwise there will be too many tests
            > that are not getting done. Is that right?

            Let me try.

            If your team is new to automated testing, it is likely that most
            every test you are now doing should be automated. The reason is that
            if you are testing manually, you are probably doing very few tests
            that only a human can do.

            So my advice would be to err on the side of over-automating for a
            while.

            Ron Jeffries
            www.XProgramming.com
            www.xprogramming.com/blog
            [S]oftware engineering ... "How to program if you cannot." -- Edsger Dijkstra
          • Don MacIntyre
            I describe the need for the adoption of automated testing and other XP practices to new Scrum teams something like this... If you don t adopt the practices
            Message 5 of 7 , Oct 5, 2010
            • 0 Attachment
              I describe the need for the adoption of automated testing and other XP practices to new Scrum teams something like this...

              If you don't adopt the practices that are best suited for iterative development, you may experience an initial burst in productivity, but you will likely soon discover that you are just building crap faster than you were before.

              I feel that continuous integration and automated testing are necessary for iterative development.

              You can do typical unit testing and have your CI system kick off other automated suites (we currently use selenium) but the biggest bang for the buck is Test Driven Development.

              My current teams have been slow to totally embrace TDD, so I'm making a real push now with weekly how-to sessions. One of the guys will also spend the next 3 days at Ron and Chet's Agile Skills class. Hopefully that will be enough to drive it home. ;)

              -don
              http://www.scrumalliance.org/profiles/26043-don-macintyre

              --- In scrumdevelopment@yahoogroups.com, Ron Jeffries <ronjeffries@...> wrote:
              >
              > Hello, Adam. On Monday, October 4, 2010, at 2:46:42 PM, you wrote:
              >
              > >> It does
              > >> mean, however, that if a team is new to automated testing, almost
              > >> every test they are not doing probably should be automated, because
              > >> they will be so far behind that they are likely not doing very much
              > >> really good testing.
              > >>
              >
              > > I had to read this sentence several times to make sure I parsed it
              > > correctly. I'll say what I think you mean, and then you can correct me
              > > ;-)
              >
              > > I think you are saying that any test that is not being done manually
              > > should be automated, because otherwise there will be too many tests
              > > that are not getting done. Is that right?
              >
              > Let me try.
              >
              > If your team is new to automated testing, it is likely that most
              > every test you are now doing should be automated. The reason is that
              > if you are testing manually, you are probably doing very few tests
              > that only a human can do.
              >
              > So my advice would be to err on the side of over-automating for a
              > while.
              >
              > Ron Jeffries
              > www.XProgramming.com
              > www.xprogramming.com/blog
              > [S]oftware engineering ... "How to program if you cannot." -- Edsger Dijkstra
              >
            • woynam
              Perhaps I work in a different type of IT shop than the typical developer, but I m really astonished that in 2010 someone can suggest that we can live without
              Message 6 of 7 , Oct 5, 2010
              • 0 Attachment
                Perhaps I work in a different type of IT shop than the typical developer, but I'm really astonished that in 2010 someone can suggest that we can live without automated tests. (The comment is not directed at you, Don.)

                Our system consists of over 2 million lines of code. A single test plan for a single complex feature takes ~70 pages to print, given that there are hundreds of variations.

                If you printed the entire set of tests (single sided), it would be over 6 feet tall. I know, we did it once for fun, and to provide a important visual clue. :-)

                Before we automated the vast majority of the tests, it would take over a month to manually execute the test suite.

                Now, how on earth can someone suggest that they're doing agile development, delivering working software every few weeks, when the execution of manual tests alone would account for weeks of time?

                On top of that, the vast majority of (regression) tests do not fail when they're run each the end of each iteration. What a colossal waste of time it would be to have your talented team poking away at the system, when they could be adding value elsewhere.

                Mark


                --- In scrumdevelopment@yahoogroups.com, "Don MacIntyre" <don@...> wrote:
                >
                > I describe the need for the adoption of automated testing and other XP practices to new Scrum teams something like this...
                >
                > If you don't adopt the practices that are best suited for iterative development, you may experience an initial burst in productivity, but you will likely soon discover that you are just building crap faster than you were before.
                >
                > I feel that continuous integration and automated testing are necessary for iterative development.
                >
                > You can do typical unit testing and have your CI system kick off other automated suites (we currently use selenium) but the biggest bang for the buck is Test Driven Development.
                >
                > My current teams have been slow to totally embrace TDD, so I'm making a real push now with weekly how-to sessions. One of the guys will also spend the next 3 days at Ron and Chet's Agile Skills class. Hopefully that will be enough to drive it home. ;)
                >
                > -don
                > http://www.scrumalliance.org/profiles/26043-don-macintyre
                >
                > --- In scrumdevelopment@yahoogroups.com, Ron Jeffries <ronjeffries@> wrote:
                > >
                > > Hello, Adam. On Monday, October 4, 2010, at 2:46:42 PM, you wrote:
                > >
                > > >> It does
                > > >> mean, however, that if a team is new to automated testing, almost
                > > >> every test they are not doing probably should be automated, because
                > > >> they will be so far behind that they are likely not doing very much
                > > >> really good testing.
                > > >>
                > >
                > > > I had to read this sentence several times to make sure I parsed it
                > > > correctly. I'll say what I think you mean, and then you can correct me
                > > > ;-)
                > >
                > > > I think you are saying that any test that is not being done manually
                > > > should be automated, because otherwise there will be too many tests
                > > > that are not getting done. Is that right?
                > >
                > > Let me try.
                > >
                > > If your team is new to automated testing, it is likely that most
                > > every test you are now doing should be automated. The reason is that
                > > if you are testing manually, you are probably doing very few tests
                > > that only a human can do.
                > >
                > > So my advice would be to err on the side of over-automating for a
                > > while.
                > >
                > > Ron Jeffries
                > > www.XProgramming.com
                > > www.xprogramming.com/blog
                > > [S]oftware engineering ... "How to program if you cannot." -- Edsger Dijkstra
                > >
                >
              Your message has been successfully submitted and would be delivered to recipients shortly.