Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Difference between top level Unit Test and Acceptance Test

Expand Messages
  • J. B. Rainsberger
    ... I would add ...for that feature... , but yes. I expect all ATs for a given feature to pass before we can claim the feature is done. (Necessary, but not
    Message 1 of 60 , Aug 31, 2005
    • 0 Attachment
      David Roberts wrote:

      > Is the rule acceptance tests must pass 100% once the feature is
      > complete?

      I would add "...for that feature...", but yes. I expect all ATs for a
      given feature to pass before we can claim the feature is done.
      (Necessary, but not sufficient condition.)

      > Currently we fail our builds if an acceptance test fails; I prefer they
      > don't check in the test until it's complete.

      I don't like this approach, because it is natural for the team always to
      have a few failing ATs; otherwise when they catch up (100% ATs pass)
      they don't know where to go next.
      --
      J. B. (Joe) Rainsberger
      Diaspar Software Services
      http://www.diasparsoftware.com
      2005 Gordon Pask Award Winner for contribution to Agile practice
      Author, JUnit Recipes: Practical Methods for Programmer Testing
    • Ron Jeffries
      ... Yes, well. I d like to address two different ways that ATs can fail. 1. An AT can fail because the story isn t done yet. Those ATs shouldn t be in the
      Message 60 of 60 , Sep 1 5:22 AM
      • 0 Attachment
        On Wednesday, August 31, 2005, at 2:57:08 PM, J. B. Rainsberger wrote:

        > David Roberts wrote:

        >> Is the rule acceptance tests must pass 100% once the feature is
        >> complete?

        > I would add "...for that feature...", but yes. I expect all ATs for a
        > given feature to pass before we can claim the feature is done.
        > (Necessary, but not sufficient condition.)

        >> Currently we fail our builds if an acceptance test fails; I prefer they
        >> don't check in the test until it's complete.

        > I don't like this approach, because it is natural for the team always to
        > have a few failing ATs; otherwise when they catch up (100% ATs pass)
        > they don't know where to go next.

        Yes, well. I'd like to address two different ways that ATs can fail.

        1. An AT can fail because the story isn't done yet. Those ATs
        shouldn't be in the build tests, I'd think, because if they were,
        the build would never run.

        2. An AT can fail after the story is done, because something we just
        did has broken it. I'd think that that would be a "deviation" and
        that we'd want to know about it.

        Therefore, I suspect that a Pretty Good Practice is to put ATs into
        the build once the story is done, and fail the build if they break.

        If that fails the build too often, we might want to look upstream of
        the build and see whether there's a firewall of some kind we would
        like to put in there.

        Ron Jeffries
        www.XProgramming.com
        New and stirring things are belittled because if they are not belittled,
        the humiliating question arises, "Why then are you not taking part in
        them?" -- H. G. Wells
      Your message has been successfully submitted and would be delivered to recipients shortly.