Loading ...
Sorry, an error occurred while loading the content.

RE: [XP] Frameworks for acceptance testing web apps.

Expand Messages
  • Joseph Graves
    At least they re using WinRunner. Count your blessings. Pros of WinRunner: existing skill set easy browser integration easy for users to define tests in user
    Message 1 of 4 , Apr 27, 2004
    • 0 Attachment
      At least they're using WinRunner. Count your blessings.

      Pros of WinRunner:
      existing skill set
      easy browser integration
      easy for users to define tests in user concepts

      Cons of WinRunner
      requires existing working application to define tests
      requires babysitting of test execution (even if steps of test are automated)
      requires steady-state on client test-execution machine

      If they have been successful with automating tests with WinRunner and have policies for automated regression test execution and result reporting, you're ahead of the game. Getting the tests defined up front is the one advantage to not using WinRunner, but you might be able to mock up target screens that the test must see to pass.

      My experience with Mercury products (QuickTestPro and TestDeveloper) is that the tests are defined and logged OK but that automation becomes a headache. Steps fail for machine configuration issues and the tester has to manually pass the failing sets, requiring a lot of intervention. I would not recommend starting from scratch with these tools, but if you have the on-staff experience then you might have enough success to warrant using them on a go-forward basis.

      My experience with roll-your-own and open-source tools (FitNesse and Avignon) are that it is difficult to get the tests written by testers/customers. They tend to need a good bit of handholding because of the scripting language. Also I have not yet seen (to date) a top-notch level of browser integration so it becomes difficult to include the fickleness of the browser in the test.

      If you were starting from scratch I would recommend training your testers in language and concepts one of the open source test tools so that they could get involved at story-definition time and help review requirements with the customers. Then they could become the people capable of designing the tests.


      -----Original Message-----
      From: daveflows [mailto:davefellows@...]
      Sent: Tue 4/27/2004 3:44 PM
      To: extremeprogramming@yahoogroups.com
      Cc:
      Subject: [XP] Frameworks for acceptance testing web apps.



      There's a debate raging at my company about the merrits of using
      particular tools as the basis of automated Acceptance Tests. The
      development community (who are working on XP based projects
      using .Net) favour writing Acceptance Tests as part of the Test
      Driven Development cycle using tools such as FIT or FAT. However as
      a legacy of the companies previous life as a waterfall shop we also
      has a large group of 'Testers' that test applications prior to them
      going live. Their preferance leans towards a tool written by Mercury
      called WinRunner in which they have a great deal of vested skills...

      Open Source Tool's supported by developers or commercial tool
      supported by testers - Views appreciated....



      To Post a message, send it to: extremeprogramming@...

      To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...

      ad-free courtesy of objectmentor.com
      Yahoo! Groups Links









      [Non-text portions of this message have been removed]
    • yahoogroups@jhrothjr.com
      From: Joseph Graves Sent: Tuesday, April 27, 2004 5:10 PM Subject: RE: [XP] Frameworks for acceptance
      Message 2 of 4 , Apr 27, 2004
      • 0 Attachment
        From: "Joseph Graves" <jgraves.at.nolacom.com@...>
        Sent: Tuesday, April 27, 2004 5:10 PM
        Subject: RE: [XP] Frameworks for acceptance testing web apps.


        [comment on WinRunner]

        > My experience with roll-your-own and open-source tools (FitNesse and
        Avignon) are that it is difficult to get the tests written by
        testers/customers. They tend to need a good bit of handholding because of
        the scripting language. Also I have not yet seen (to date) a top-notch level
        of browser integration so it becomes difficult to include the fickleness of
        the browser in the test.

        I'm not personally familiar with Avignon, but a quick look shows
        that it's an XML based product. I wouldn't hand that to a customer
        unless the customer was fairly technically inclined. It's also Java
        centric.

        FitNesse is based on FIT, and FIT tests can be written using tables in
        familiar tools such as Word and Excel. That's by design. I personally
        don't like Wiki style markup - it's just one more set of arbitrary line
        noise I have
        to memorize to get the job done.

        On the other hand, I agree with the hand-holding, even without the line
        noise. It takes a couple (and sometimes more than a couple) of walkthroughs
        before the light dawns on exactly how an acceptance test relates to a story,
        and how that in turn relates to the application, and how the application's
        design affects the tests and the tests affect the application's design.

        > If you were starting from scratch I would recommend training your testers
        in
        language and concepts one of the open source test tools so that they could
        get
        involved at story-definition time and help review requirements with the
        customers.
        Then they could become the people capable of designing the tests.

        I'd go for having the customers more involved.

        John Roth
        >
        >
        > -----Original Message-----
        > From: daveflows [mailto:davefellows@...]
        > Sent: Tue 4/27/2004 3:44 PM
        > To: extremeprogramming@yahoogroups.com
        > Cc:
        > Subject: [XP] Frameworks for acceptance testing web apps.
        >
        >
        >
        > There's a debate raging at my company about the merrits of using
        > particular tools as the basis of automated Acceptance Tests. The
        > development community (who are working on XP based projects
        > using .Net) favour writing Acceptance Tests as part of the Test
        > Driven Development cycle using tools such as FIT or FAT. However as
        > a legacy of the companies previous life as a waterfall shop we also
        > has a large group of 'Testers' that test applications prior to them
        > going live. Their preferance leans towards a tool written by Mercury
        > called WinRunner in which they have a great deal of vested skills...
        >
        > Open Source Tool's supported by developers or commercial tool
        > supported by testers - Views appreciated....
        >
        >
        >
        > To Post a message, send it to: extremeprogramming@...
        >
        > To Unsubscribe, send a blank message to:
        extremeprogramming-unsubscribe@...
        >
        > ad-free courtesy of objectmentor.com
        > Yahoo! Groups Links
        >
        >
        >
        >
        >
        >
        >
        >
        >
        > [Non-text portions of this message have been removed]
        >
        >
        >
        > To Post a message, send it to: extremeprogramming@...
        >
        > To Unsubscribe, send a blank message to:
        extremeprogramming-unsubscribe@...
        >
        > ad-free courtesy of objectmentor.com
        > Yahoo! Groups Links
        >
        >
        >
        >
        >
      • Chris Hanson
        ... This is the recommendation I ve heard from several people, and it s what I d try. The QA people who used to be at the end of the process move some of
        Message 3 of 4 , May 1, 2004
        • 0 Attachment
          On Apr 27, 2004, at 2:10 PM, Joseph Graves wrote:
          > If you were starting from scratch I would recommend training your
          > testers in language and concepts one of the open source test tools so
          > that they could get involved at story-definition time and help review
          > requirements with the customers. Then they could become the people
          > capable of designing the tests.

          This is the recommendation I've heard from several people, and it's
          what I'd try. The QA people who used to be at the end of the process
          move some of their work to the start of the process, helping the
          customers to specify -- in the form of tests -- the acceptance criteria
          for individual user stories.

          As Robert C. Martin says, "Testing is a form of specification, not
          verification."

          -- Chris

          --
          Chris Hanson <cmh@...>
          http://www.livejournal.com/users/chanson/
        • Steven J. Owens
          ... A good way to pitch this might be: Q: What s the purpose of testing? A: To make sure the software does what it s supposed to do. Q: How do you know what
          Message 4 of 4 , May 2, 2004
          • 0 Attachment
            On Sat, May 01, 2004 at 11:51:14PM -0700, Chris Hanson wrote:
            > This is the recommendation I've heard from several people, and it's
            > what I'd try. The QA people who used to be at the end of the process
            > move some of their work to the start of the process, helping the
            > customers to specify -- in the form of tests -- the acceptance criteria
            > for individual user stories.
            >
            > As Robert C. Martin says, "Testing is a form of specification, not
            > verification."

            A good way to pitch this might be:

            Q: What's the purpose of testing?
            A: To make sure the software does what it's supposed to do.

            Q: How do you know what it's supposed to do?
            A: Get involved with the customer early to help them specify it.

            ...leading ultimately to the "specification, not verification"
            point of view.


            --
            Steven J. Owens
            puff@...

            "I'm going to make broad, sweeping generalizations and strong,
            declarative statements, because otherwise I'll be here all night and
            this document will be four times longer and much less fun to read.
            Take it all with a grain of salt." - Me at http://darksleep.com
          Your message has been successfully submitted and would be delivered to recipients shortly.