Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] FIT is bad for automation (was How are you doing 1-week iterations?)

Expand Messages
  • Rick Mugridge
    Hi Michael, I disagree with James that Fit is bad for automation. (He s limited his use and advocacy to part of the original core Fit, by the way.) Sure
    Message 1 of 51 , Dec 1, 2007
    • 0 Attachment
      Hi Michael,

      I disagree with James that Fit is bad for automation. (He's limited his
      use and advocacy to part of the original core Fit, by the way.) Sure
      there's scope for improvement, as always, and it's easy to get caught up
      in unnecessary fixturing. I'm working on the future evolution of the Fit
      ideas to improve upon them. Refactoring of storytests and into the code
      is an important part of this. Another is in evolving the "language" to
      improve expressiveness and to avoid fixturing overhead. Another is in
      handling the range from surface-level tests (such as Selenium) to
      essential tests, such as the DomainFixture style of FitLibrary.

      Selenium is good for testing the ui, but Selenium tests require detailed
      knowledge of the ui to write (or record). I've found them to be good for
      testing complex pieces of ui, such as with ajax, plus general page
      traversal.

      But such tests can get in the way if the UI is evolving. Maintaining
      this style of test has become a major problem for many organisations
      (although there are good solutions). And they are very slow to run. I
      want my acceptance tests to be part of the build and I want the build to
      run in a few minutes. So I use Fit/FitLibrary for testing into the
      domain layer, and specify/test all of the business logic at that level.
      It's worked well for me and many teams who are using it effectively.

      I also find that surface-level tests don't play much role in
      person-to-person communication in a team about the domain. Much of my
      experience is in more complex domains where such discussions and
      refinement of thinking using such examples are very important. Simple
      CRUD-style systems are much less likely to get such value from storytests.

      How complex is your domain? How are you finding, with Selenium tests:

      1) Evolving the tests (or is your ui very stable)?
      2) Speed of running (and the impact on feedback time)?
      3) Their value in thinking about the domain?

      Cheers, Rick

      Michael Dubakov wrote:
      >
      >
      > ...
      >
      > But we do write regression tests to make sure that all working as
      > expected. And regression tests on Selenium are MUCH easier to create
      > and support.
      >
      > Michael Dubakov
      > http://www.targetprocess.com <http://www.targetprocess.com>
      >
      >


      [Non-text portions of this message have been removed]
    • Simon Jones
      ... Reasonably complex in my previous incarnation. One internal product management system and a publicly facing corporate webshop and customer
      Message 51 of 51 , Dec 3, 2007
      • 0 Attachment
        <snip>

        > How complex is your domain? How are you finding, with Selenium
        tests:
        >
        Reasonably complex in my previous incarnation. One internal product
        management system and a publicly facing corporate webshop and
        customer self-serve.

        > 1) Evolving the tests (or is your ui very stable)?
        Not too bad, but, as the UI changed quite a bit Selenium tests can
        become quite fragile. One of the ways we mitigated this was to use
        another web system (OpenACS) as a test producer. Because its TCL
        based manipulating text and creating dynamic pages is easy, so often
        Selenium tests became TCL web pages designed to dynamically generate
        the required selenium based on querying the application.

        > 2) Speed of running (and the impact on feedback time)?
        Can be slow yes.. Using firefox and a few tweaks helped though.

        > 3) Their value in thinking about the domain?
        This is where I think selenium wins and also where its useful when
        the UI is actually very important.

        Selenium is technical enough that you can do quite powerful things
        with it. Its simple enough that any tester can pick it up quite
        easily and its /visible/ enough that the customer can /see/ what the
        test is doing.

        Although Selenese tests can become quite time consuming to manage, in
        our case it was worth it becuase such importance was placed on the
        behaviour and appearance of the website under test. Things like
        whether certain text appeared, certain HTMl directives were in the
        page, certain flows behaved as expected were all quite high priority
        for our customer.

        But, from a personal POV I would highly recommend OpenACS as a pretty
        neat 'testing platform'. The TCL driven architecture (and AOLServer)
        were pretty much ahead of their time a few years ago, but they still
        perform quite well and most importantly its easy to build web stuff
        quickly... TCL always was a pretty good language for test
        development, even before agile.
        >
        > Cheers, Rick
        >
        > Michael Dubakov wrote:
        > >
        > >
        > > ...
        > >
        > > But we do write regression tests to make sure that all working as
        > > expected. And regression tests on Selenium are MUCH easier to
        create
        > > and support.
        > >
        > > Michael Dubakov
        > > http://www.targetprocess.com <http://www.targetprocess.com>
        > >
        > >
        >
        >
        > [Non-text portions of this message have been removed]
        >
      Your message has been successfully submitted and would be delivered to recipients shortly.