Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] The 'Extreme Programming' Testing Trap

Expand Messages
  • Andrew Forward
    Just a few comments.... so that by the tenth iteration there is at least 10X the amount of testing to be done Yes. But, does that mean 10X the time relative
    Message 1 of 11 , Jan 29, 2003
    • 0 Attachment
      Just a few comments....

      "so that by the tenth iteration there is at least 10X the amount of testing
      to be done"

      Yes. But, does that mean 10X the time relative to writing the code? Not
      really. With an automated test suite testing 100X can be accomplished with
      a simple push of the button.

      Of course, this may hold true in a non-test automated environment.


      "such as the limited seat count - - was assumed to still work since it did
      before"

      Using TDD, there are fewer assumptions because of the supporting test-suite
      making sure the assertions you had made previous to the new feature continue
      to hold.

      Of course, this may hold true in a non-test automated environment.


      "Here's an example"

      An XP example? I doubt it... the way in which this airline project
      approached testing seems quite different than that promoted by xp practices.


      "Unfortunately, this meant the only thing that could be tested was whether
      the eligibility rules worked"

      Signs of a non-TDD project. Why is the only thing that could be tested was
      the eligibility rules?


      "why rapid development doesn't necessarily translate into rapid testing"

      Development in a XP manner involves testing... therefore if a project is
      able to rapidly develop software - following XP / TDD practices - then a
      welcomed side-product is rapid testing.

      Overall - the articles seems more pointed at code and debug style
      development and not really XP (or at least coding processes that are rapid
      but with little concern for automating testing). XP does not solely mean
      developing software in a rapid frenzy... in my view, there is a lot of
      process involved in XP - it's just that because we take such small (but
      meaningful and productive) steps it seems like rapid development because
      fewer unforeseen problems (such as those stated in the article) occur.

      andrew f


      ----- Original Message -----
      From: <sartin@...>
      To: <extremeprogramming@yahoogroups.com>
      Sent: Wednesday, January 29, 2003 12:55 PM
      Subject: [XP] The 'Extreme Programming' Testing Trap


      > http://itmanagement.earthweb.com/columns/quaquest/article.php/1576121
      >
      > This article seems to ignore the reality of unit testing and the
      > possibility of test automation in all phases of testing. I haven't
      > yet composed a response. I thought some people here might do better
      > than I would as I tend to lose diplomacy when I detect strawman
      > arguments like this one.
      >
      > Regards,
      >
      > Rob
      >
      >
      >
      > To Post a message, send it to: extremeprogramming@...
      >
      > To Unsubscribe, send a blank message to:
      extremeprogramming-unsubscribe@...
      >
      > ad-free courtesy of objectmentor.com
      >
      > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
      >
    • Kevin Lawrence <kevin@diamond-sky.com>
      http://itmanagement.earthweb.com/columns/quaquest/article.php/1576121 ... I am sure the author knows all about automated testing since Linda was largely
      Message 2 of 11 , Jan 29, 2003
      • 0 Attachment
        http://itmanagement.earthweb.com/columns/quaquest/article.php/1576121
        >
        > This article seems to ignore the reality of unit testing and the
        > possibility of test automation in all phases of testing.

        I am sure the author knows all about automated testing since

        "Linda was largely responsible for launching and building the
        automated testing industry" -
        http://www.worksoft.com/corporate/index.asp
      • jeffgrigg63132 <jgrigg@mo.net>
        ... I just exchanged notes with the author. (She s on vacation this week, and will be more available for discussion until next week.) She says that she d like
        Message 3 of 11 , Jan 30, 2003
        • 0 Attachment
          --- William Pietri <william@s...> wrote:
          > I note that if you take away the title, the article seems
          > much more reasonable. [...]

          I just exchanged notes with the author. (She's on vacation this
          week, and will be more available for discussion until next week.)

          She says that she'd like to hear more about my positive experiences
          with XP, and that she may write about it.


          It seems that her experience is primarily with web
          apps. ...apparently using 3rd party tools that generate the HTML.
          Tools which, for some unknown reason, have a habit of changing all
          the "id" names of the components, breaking all GUI-driving tests.

          I can see how she'd feel "burnt" by this experience.

          OTOH, my current project happens to be a web app. (JSP pages) We
          don't have any particular "id" changing problems, as long as we
          maintain awareness of the issue. (...which lapsed earlier this
          week, and we happened to talk about it this morning. ;-)
        • Phlip
          ... Hmmm. TestFirstVsWizards. It seems until someone invents a Wizard that can react to a failing test by adding code (thereby promoting the Wizard to a
          Message 4 of 11 , Jan 30, 2003
          • 0 Attachment
            "jeffgrigg63132 sez:

            > It seems that her experience is primarily with web
            > apps. ...apparently using 3rd party tools that generate the HTML.
            > Tools which, for some unknown reason, have a habit of changing all
            > the "id" names of the components, breaking all GUI-driving tests.

            Hmmm. TestFirstVsWizards.

            It seems until someone invents a Wizard that can react to a failing test by
            adding code (thereby promoting the Wizard to a language by itself), we are
            stuck with not using the 'id's when we test HTML. Interesting.

            > OTOH, my current project happens to be a web app. (JSP pages) We
            > don't have any particular "id" changing problems, as long as we
            > maintain awareness of the issue. (...which lapsed earlier this
            > week, and we happened to talk about it this morning. ;-)

            Do you test using other than the 'id's? What do your scripts look like?

            Can you use these tests in a true TDD scenario?

            --
            Phlip
            http://c2.com/cgi/wiki?SheChangeDesignInTheDatabase
            -- Have a :-) day --
          • William Pietri
            ... There are many things that they could match on: * id attribute * name attribute * link url * link text * alt text * position relative to other page
            Message 5 of 11 , Jan 31, 2003
            • 0 Attachment
              On Thu, 2003-01-30 at 19:14, jeffgrigg63132 wrote:
              >
              > It seems that her experience is primarily with web
              > apps. ...apparently using 3rd party tools that generate the HTML.
              > Tools which, for some unknown reason, have a habit of changing all
              > the "id" names of the components, breaking all GUI-driving tests.

              There are many things that they could match on:
              * id attribute
              * name attribute
              * link url
              * link text
              * alt text
              * position relative to other page elements (e.g., extract the
              navbar, take the 3rd link)

              And they could extend this further by adding a little pattern matching,
              e.g., doing caseless searching for a link that contains the word "next",
              whether that's "next page", "Next Item", etc.


              So if this was an XP project, I'd be curious to hear why the developers
              weren't taking responsibility for their choice of 3rd party tools and
              either fixing the HTML output or giving the testers the needed
              assistance. I've done some of these things in an HttpUnit wrapper, and
              it's not hard.

              William

              --
              brains for sale: http://scissor.com/
            • jeffgrigg63132 <jgrigg@mo.net>
              ... Rational Robot. I m told that it uses the id attributes of the HTML controls as the basis for all testing. I m pretty sure that one can write code in
              Message 6 of 11 , Jan 31, 2003
              • 0 Attachment
                > Jeff Grigg said:
                >> My current project happens to be a web app. (JSP pages)
                >> We don't have any particular "id" changing problems, as
                >> long as we maintain awareness of the issue. (...which
                >> lapsed earlier this week, and we happened to talk about
                >> it this morning. ;-)

                --- Phlip <plumlee@s...> wrote:
                > Do you test using other than the 'id's?
                > What do your scripts look like?

                Rational Robot. I'm told that it uses the "id" attributes of the
                HTML controls as the basis for all testing.

                I'm pretty sure that one can write code in Robot to find and test
                controls based on more heuristic criteria. But the wizards wouldn't
                do that for you, and there would be issues with reliability and
                maintainability.

                > Can you use these tests in a true TDD scenario?

                My experience on multiple projects has been that Rational Robot is
                so expensive that most organizations restrict its use to only
                members of an external professional QA organization. I've yet to
                see such usage do a project any good.

                But I have hope! ;->
                - jeff

                "Hope springs eternal."
              Your message has been successfully submitted and would be delivered to recipients shortly.