Loading ...
Sorry, an error occurred while loading the content.

RFC: Acceptance Test Tools

Expand Messages
  • Thomas Robbs
    I m currently looking for an Acceptance Test tool which will help our team build automated acceptance tests. A user accesses and uses our product by way of a
    Message 1 of 7 , Oct 14, 2003
    • 0 Attachment
      I'm currently looking for an Acceptance Test tool which will help our
      team build automated acceptance tests. A user accesses and uses our
      product by way of a web browser, however the product may be deployed in
      an application server in the operator's NOC or on a CPE device in the
      home. Either way, the user still uses a web browser to use the services
      provided by our software. This is the first area which I'll focus
      acceptance testing on. Other components of our solution could be
      treated as individual products and could require a different approach.
      I'll worry about that as the need arises. :)

      With that in mind, and with me having just read that there was an update
      to http://www.xprogramming.com/software.htm, I've noticed new acceptance
      test tools are available! I'd like to survey the group to get a general
      feel for some of these tools to make my evaluation process a little
      faster; stand on the shoulders of giants as it were.

      So, can anyone comment on the following tools and how easy they are to
      use, what type of people can use them, which would best suit testing a
      web-based application, etc? I'll likely be downloading each of them for
      a quick trial, but having the insight of more seasoned veterans always
      makes me feel a little more comfortable about the "long view" for these
      tools.

      Without further adeu...

      FIT: http://fit.c2.com/
      Avignon Acceptance Testing: http://www.nolacom.com/avignon/index.asp
      FitNesse Acceptance Testing: http://fitnesse.org/
      ITP: http://www.incanica.com/itp.html

      They all sound similar in approach, so I'm hoping that there will be
      some obvious differentators to help me make my choice.

      Thanks in advance to the giants... :)

      Cheers,

      Thomas Robbs
      Project Lead
      Casero Inc.

      E: thomas.robbs@...
      W: +1.647.439.0102, x4118
      F: +1.647.439.0108
    • Curtis Cooley
      ... I ve had success testing web applications with fitnesse, WATT, and jWebFit (a plugin supplied with jWebUnit). I ve used the fitnesse wiki interface and
      Message 2 of 7 , Oct 14, 2003
      • 0 Attachment
        Thomas Robbs wrote:
        > FIT: http://fit.c2.com/
        > Avignon Acceptance Testing: http://www.nolacom.com/avignon/index.asp
        > FitNesse Acceptance Testing: http://fitnesse.org/
        > ITP: http://www.incanica.com/itp.html
        >
        I've had success testing web applications with fitnesse, WATT, and
        jWebFit (a plugin supplied with jWebUnit). I've used the fitnesse wiki
        interface and WATT to write tests to make sure pages are assembled as
        expected. I've use fitenesse and jWebFit to submit forms and ensure that
        I've gotten back what I expect.

        The documentation for jWebFit is sparse, and you have to grab the latest
        version from jWebUnit's cvs repository, but once I got it working, it
        was sure nice to be able to build acceptance tests using nothing but
        wiki tables.
        --
        Curtis R Cooley
        curtis@...
        ------------------
        Hardware is indeterministically reliable.
        Software is deterministically unreliable.
        People are indeterministically unreliable.
        Nature is deterministically reliable.
      • Thomas Robbs
        ... Just to provide my feedback into the loop, I m not sure I feel comfortable with the level of redirection here for a customer who would (hopefully) be
        Message 3 of 7 , Oct 14, 2003
        • 0 Attachment
          > -----Original Message-----
          > From: Curtis Cooley [mailto:curtis@...]
          >
          > Thomas Robbs wrote:
          > > FIT: http://fit.c2.com/
          > > Avignon Acceptance Testing: http://www.nolacom.com/avignon/index.asp
          > > FitNesse Acceptance Testing: http://fitnesse.org/
          > > ITP: http://www.incanica.com/itp.html
          > >
          > I've had success testing web applications with fitnesse, WATT, and
          > jWebFit (a plugin supplied with jWebUnit). I've used the
          > fitnesse wiki
          > interface and WATT to write tests to make sure pages are assembled as
          > expected. I've use fitenesse and jWebFit to submit forms and
          > ensure that
          > I've gotten back what I expect.
          >
          > The documentation for jWebFit is sparse, and you have to grab
          > the latest
          > version from jWebUnit's cvs repository, but once I got it working, it
          > was sure nice to be able to build acceptance tests using nothing but
          > wiki tables.

          Just to provide my feedback into the loop, I'm not sure I feel
          comfortable with the level of redirection here for a customer who would
          (hopefully) be writing the tests, i.e. Wiki Tables.

          I'm a Wiki-newb so maybe it's easier than I think. Regardless, I'll add
          the tools to my list.

          I think I should add another comment that will help to quantify the
          solution that I'm looking for... <submitting user story> I'm primarily
          focused with exercising the application's functionality with little
          concern to the rendering of the returned values in the browser. I'm
          assuming, at this point, that the browser will render the data properly.
          Thus, I'll build an acceptance of GoodData (i.e. data that renders in
          our supported browsers properly) through acceptance tests without a
          browser.

          For the time being, the human eye can validate that the Data renders
          properly in each supported browser. It won't benefit us (vs. cost) to
          work directly with the supported browsers as we're only supporting one
          (officially) for the time being (IE) and the customer will be doing
          their own acceptance testing outside of our test framework, i.e. manual
          testing and building their own acceptance tests in their own framework
          (their choice).

          It's late in the day and I hope those last two paragraphs made sense.
          Feel free to beat me with the idiot stick and I'll try and compose a
          more succinct message in the morn. ;)

          Thanks again for the input!

          Cheers,

          Tom
        • Joshua Kerievsky
          We ve been using Ward s FIT for about a year now with many of our clients. It rocks. FIT makes it easy for storytellers (people who write stories) to
          Message 4 of 7 , Oct 14, 2003
          • 0 Attachment
            We've been using Ward's FIT for about a year now with many of our
            clients. It rocks.

            FIT makes it easy for storytellers (people who write stories) to
            collaborate with others (analysts, testers, QA folks, programmers) to
            produce storytests (e.g. FIT tables and explanatory text, soon to be
            automated), which then get automated via story test-driven development
            (in which you don't write a line of code until you've automated a
            failing storytest).

            I believe Ward is doing a FIT tutorial at OOPSLA.

            We've also used Fitnesse (an integration of FIT and Wiki) for over 6
            months now. We like it very much as well.

            best regards,
            jk
          • yahoogroups@jhrothjr.com
            ... From: Thomas Robbs To: extremeprogramming@yahoogroups.com
            Message 5 of 7 , Oct 14, 2003
            • 0 Attachment
              ----- Original Message -----
              From: "Thomas Robbs"
              <thomas.robbs.at.casero.com@...>
              To: "extremeprogramming@yahoogroups.com"
              <extremeprogramming.at.yahoogroups.com@...>
              Sent: Tuesday, October 14, 2003 3:48 PM
              Subject: [XP] RFC: Acceptance Test Tools


              >
              > I'm currently looking for an Acceptance Test tool which will help our
              > team build automated acceptance tests. A user accesses and uses our
              > product by way of a web browser, however the product may be deployed in
              > an application server in the operator's NOC or on a CPE device in the
              > home. Either way, the user still uses a web browser to use the services
              > provided by our software. This is the first area which I'll focus
              > acceptance testing on. Other components of our solution could be
              > treated as individual products and could require a different approach.
              > I'll worry about that as the need arises. :)
              >
              > With that in mind, and with me having just read that there was an update
              > to http://www.xprogramming.com/software.htm, I've noticed new acceptance
              > test tools are available! I'd like to survey the group to get a general
              > feel for some of these tools to make my evaluation process a little
              > faster; stand on the shoulders of giants as it were.
              >
              > So, can anyone comment on the following tools and how easy they are to
              > use, what type of people can use them, which would best suit testing a
              > web-based application, etc? I'll likely be downloading each of them for
              > a quick trial, but having the insight of more seasoned veterans always
              > makes me feel a little more comfortable about the "long view" for these
              > tools.
              >
              > Without further adeu...
              >
              > FIT: http://fit.c2.com/
              > Avignon Acceptance Testing: http://www.nolacom.com/avignon/index.asp
              > FitNesse Acceptance Testing: http://fitnesse.org/
              > ITP: http://www.incanica.com/itp.html
              >
              > They all sound similar in approach, so I'm hoping that there will be
              > some obvious differentators to help me make my choice.
              >
              > Thanks in advance to the giants... :)

              FIT rocks, FitNesse is a Wiki style wrapper around FIT that has
              some additional management tools for program development
              built in. You can use FIT without FitNesse.

              FIT is designed to allow you to write acceptance tests with tools your
              customer is familiar with: Word and Excel, among others. Just save
              in HTML format, and they work well.

              Whether or not you use FitNesse with FIT is more a matter of
              taste and the needs of your development organization.

              The others I'm not familiar with.

              John Roth

              >
              > Cheers,
              >
              > Thomas Robbs
              > Project Lead
              > Casero Inc.
              >
              > E: thomas.robbs@...
              > W: +1.647.439.0102, x4118
              > F: +1.647.439.0108
              >
              > To Post a message, send it to: extremeprogramming@...
              >
              > To Unsubscribe, send a blank message to:
              extremeprogramming-unsubscribe@...
              >
              > ad-free courtesy of objectmentor.com
              >
              > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
              >
              >
            • Curtis Cooley
              ... Then using fit to test the domain model behind the web interface should work nicely. Sounds like there is no domain code in the web interface, so you
              Message 6 of 7 , Oct 15, 2003
              • 0 Attachment
                Thomas Robbs wrote:
                >>-----Original Message-----
                >>From: Curtis Cooley [mailto:curtis@...]
                >>
                >>Thomas Robbs wrote:
                >> > FIT: http://fit.c2.com/
                >>
                >>>Avignon Acceptance Testing: http://www.nolacom.com/avignon/index.asp
                >>>FitNesse Acceptance Testing: http://fitnesse.org/
                >>>ITP: http://www.incanica.com/itp.html
                >>>
                >>
                >>I've had success testing web applications with fitnesse, WATT, and
                >>jWebFit (a plugin supplied with jWebUnit). I've used the
                >>fitnesse wiki
                >>interface and WATT to write tests to make sure pages are assembled as
                >>expected. I've use fitenesse and jWebFit to submit forms and
                >>ensure that
                >>I've gotten back what I expect.
                >>
                >>The documentation for jWebFit is sparse, and you have to grab
                >>the latest
                >>version from jWebUnit's cvs repository, but once I got it working, it
                >>was sure nice to be able to build acceptance tests using nothing but
                >>wiki tables.
                >
                >
                > Just to provide my feedback into the loop, I'm not sure I feel
                > comfortable with the level of redirection here for a customer who would
                > (hopefully) be writing the tests, i.e. Wiki Tables.
                >
                > I'm a Wiki-newb so maybe it's easier than I think. Regardless, I'll add
                > the tools to my list.
                >
                > I think I should add another comment that will help to quantify the
                > solution that I'm looking for... <submitting user story> I'm primarily
                > focused with exercising the application's functionality with little
                > concern to the rendering of the returned values in the browser. I'm
                > assuming, at this point, that the browser will render the data properly.
                > Thus, I'll build an acceptance of GoodData (i.e. data that renders in
                > our supported browsers properly) through acceptance tests without a
                > browser.
                >
                > For the time being, the human eye can validate that the Data renders
                > properly in each supported browser. It won't benefit us (vs. cost) to
                > work directly with the supported browsers as we're only supporting one
                > (officially) for the time being (IE) and the customer will be doing
                > their own acceptance testing outside of our test framework, i.e. manual
                > testing and building their own acceptance tests in their own framework
                > (their choice).

                Then using fit to test the domain model behind the web interface should
                work nicely. Sounds like there is no domain code in the web interface,
                so you should be able to use fit to supply data and test the results.

                My experience with jWebFit indicates that not using the web interface
                should be faster. Even a moderate number of web tests run slower and slower.

                --
                Curtis R Cooley
                curtis@...
                ------------------
                Hardware is indeterministically reliable.
                Software is deterministically unreliable.
                People are indeterministically unreliable.
                Nature is deterministically reliable.
              • jrb32002
                ... At Eidogen, we re coming at the same general set (on Linux) from a different direction. We have two components which can be driven textually (like
                Message 7 of 7 , Oct 15, 2003
                • 0 Attachment
                  --- In extremeprogramming@yahoogroups.com, "Thomas Robbs"
                  <thomas.robbs@c...> wrote:
                  >
                  > I'm currently looking for an Acceptance Test tool which will help our
                  > team build automated acceptance tests. A user accesses and uses our
                  > product by way of a web browser, however the product may be deployed in
                  > an application server in the operator's NOC or on a CPE device in the
                  > home.

                  At Eidogen, we're coming at the same general set (on Linux) from a
                  different direction. We have two components which can be driven
                  textually (like shells), one of which also has a GUI facet. We later
                  added a component which serves webpages, and there's also some
                  database construction and validation thrown into the mix.

                  I didn't see an easy way to handle all that with the tools listed,
                  so the team and I grew our own. A reimplementation under copyleft is
                  underway at http://acceptor.tigris.org/. Consider this an invitation
                  to comment and contribute. (Right now I'm blocking on how one goes
                  about documenting APIs. :-b Tips, hints, and block-busts appreciated.)

                  Joseph Beckenbach
                  lead XP tester, Eidogen Inc.
                Your message has been successfully submitted and would be delivered to recipients shortly.