Loading ...
Sorry, an error occurred while loading the content.

RE: Customer (ie Acceptance, Functional) tests

Expand Messages
  • CORUM, M E [AG/1000]
    There is some confusion about acceptance tests. I think if we look at XP over the last few years, acceptance (or customer) tests have evolved a bit in the XP
    Message 1 of 2 , Apr 27, 2004
      There is some confusion about acceptance tests. I think if we look
      at XP over the last few years, acceptance (or customer) tests have
      evolved a bit in the XP community. At first, there was a stronger
      emphasis on testing from the GUI layer. That has now morphed to a
      strong bias against testing at the GUI layer. People have to look at
      what makes sense in their situation. I agree that the general
      philosophy should be that we make the GUIs razor thin and write ATs
      at the layer just behind that layer when possible. For our applications,
      the answer is a bit different for different deployment architectures.
      For pure web apps presenting html (with javascript, etc..), it is
      possible to test at the GUI layer very well with tools like httpunit.
      Httpunit simulates the browser and does a fine job for simple web apps.
      When the web apps become more complex, it may be necessary to start
      thinking in terms of testing just behind the GUI and using something
      like jsunit to test more complex GUI interactions that involve javascript.

      The answer may be different for Java desktop apps or Java applets. This
      depends a bit on the complexity of your user interface. I'm involved with
      applications written for research scientists. In general, simple, easy to
      use user interfaces make research scientists angry. The more complex the
      better. In fact, some of the apps I work with resemble modern video games
      and the customers are trying to figure out ways to make the UIs even more
      complex. When you have the combination of Java, where a true MVC model is
      difficult to do because Java blurs the lines in Swing, and complex GUIs
      that resemble video games, it is much more difficult to not test the GUI.
      We tried and bug counts went up so we make many of our ATs start with the
      GUI. We use Java's built-in java.awt.Robot class and we've built some
      excellent toolkits around that. We've learned a lot about testing GUIs
      over the years. For instance, in some cases, it is possible to test
      sufficiently by just packing the frame and not actually making it visible.
      There are various other techniques that help as well. While we are always
      looking for ways to do less testing that starts with the GUI, most of our
      ATs (for complex Java apps) start with the GUI and that is working for us.

      We are just starting to get into ATs for C# apps. I've found that, so far,
      I've had to use Phlip's techniques (from TFUI postings). Over time, we will
      build up utilities over the win32 api's for poking at the GUI and sending
      events. That way, we will have the capability to do it both ways.

      One last thing. I'm very much opposed to using a "guru-testing-tool" like
      Rational Robot for agile projects. These tools go against the agile
      philosophy in several ways (including creating guru's who are the only ones
      that can create or run the tests) and they are way too slow to keep up with
      an agile project. I have experience with that.

      I think it would be valuable if the XP community could catalog techniques
      in three areas:
      1. Techniques for doing "sufficient" ATs without starting from the GUI.
      2. Techniques for making GUI-driven testing work well when it has to be done.
      3. How to tell when GUI-driven testing has to be done.

      My list of three makes it sound more simple than it is. Each of those sets
      of techniques can change depending on the deployment style, the language
      employed, and other issues.

      Mike Corum

      -----Original Message-----
      >Message: 12
      > Date: Mon, 26 Apr 2004 21:29:56 -0400
      > From: yahoogroups@...
      >Subject: Customer (ie Acceptance, Functional) tests
      >From: "Steven J. Owens" <xp.at.darksleep.com@...>
      >Sent: Monday, April 26, 2004 6:07 PM
      >Subject: Re: [XP] Re: Introducing XP into an organization
      >> On Mon, Apr 26, 2004 at 07:31:05AM -0700, Phlip wrote:
      >> | Your site should be invented using Test-Driven
      >> | Development on each (logical) feature, before any
      >> | acceptance tests. (And note that "acceptance test"
      >> | does not mean "test thru the UI".)
      >> Care to enlarge on this a little?
      >> A couple of times I've thought about posting a question seeking
      >> more concrete definitions of acceptance tests. I don't recall seeing
      >> an answer, or maybe this particular question didn't make the final cut
      >> in my posts: What exactly are "acceptance tests"?
      >The "proper" name for them now is "customer" tests
      >to designate the owner, or possibly "functional" tests
      >to indicate what they do.
    Your message has been successfully submitted and would be delivered to recipients shortly.