Loading ...
Sorry, an error occurred while loading the content.

[XP] Re: Unit vs Integration TDD

Expand Messages
  • Matt
    Jeff, I actually was asking Ron how to eliminate the permutations, not test them. Writing the acceptance tests is fairly easy. Except you missed a few. - All
    Message 1 of 36 , Nov 2, 2008
    • 0 Attachment
      Jeff,

      I actually was asking Ron how to eliminate the permutations, not test
      them. Writing the acceptance tests is fairly easy. Except you missed a
      few.

      - All lower case letters in good url

      - All lower case letters in a bad url

      - All capital letters in a good url

      - All capital letters in a bad url

      - Proper cased good url

      - Proper cased bad url

      - Good url with the domain in upper case

      (I think you can see where this is going)

      If you *only* test at the acceptance level, these are truly valid
      tests... my user wants the browser to handle all of these cases
      properly.

      I would rather test one of the above cases at the acceptance level and
      then write unit tests on my UrlParser class to make sure it handles
      upper / lower cases properly. Once I am confident that my URL will be
      parsed properly, I can be fairly confident that I don't need to test
      every combination at the user level.

      Matt


      --- In extremeprogramming@yahoogroups.com, "Jeff Grigg" <jeffgrigg@...>
      wrote:
      >
      > --- "Matt" maswaffer@ wrote:
      > > I suppose. Although when I entered the web address to
      > > get to my gmail [... (see below) ...]
      > >
      > > And I haven't even gotten to gmail the application yet.
      > >
      > > So how would you approach eliminating permutations?
      >
      > At first I thought we were talking about the difficulty in writing
      > and testing a web mail application like gmail. But since none of
      > these test cases have touched gmail code yet, I think we're really
      > talking about how to write acceptance integration tests to drive Test
      > Driven Development (TDD) of a browser.
      >
      > I think it's helpful to notice that none of the questions above talks
      > about anything that wouldn't be visible to the end user. That makes
      > everything mentioned, and all the details that would be needed to
      > turn these questions into tests, good material for end-to-end
      > integration tests -- automated acceptance tests.
      >
      > > if I put http:// in first or mail. or just google.com.
      >
      > Test:
      > User puts 'http://mail.google.com' into the address box.
      > Result: User gets gmail signon page.
      >
      > Test: User puts 'mail.google.com' into the address box.
      > Result: User gets gmail signon page.
      >
      > Test: User types 'mail' into the address box.
      > Result: Ajax style pop-up displays,
      > including 'http://mail.google.com' as an option.
      > Action: User selects 'http://mail.google.com' from list.
      > Result: User gets gmail signon page.
      >
      > > And then of course if I put in an invalid page it had
      > > to know what to do if I got redirected. Or just display
      > > an error message.
      >
      > Test: DNS lookup of host name fails.
      > [Note: This may require mocking DNS or using a known bad host name.]
      > Result: Appropriate informative error page with some kind of useful
      > text that may help the user figure out what to do next. [Discuss the
      > issue with some User Interface experts if you don't know what text
      > would be helpful to end users, given how your programs user interface
      > is designed.]
      >
      > Test: DNS lookup succeeds, but attempts to contact the host at that
      > IP address time out.
      > [Note: May require mocking DNS and/or network layer]
      > Result: Display an appropriate error message page.
      >
      > Test: Host returns an HTTP standard redirect error, with a valid URL
      > to redirect to.
      > Result: The "redirect to" page is fetched and displayed.
      > [Note: One would probably need test web server(s) for this kind of
      > testing. I would probably favor mocking the network layer and
      > simulating the operation of the remote web servers.]
      >
      > > Or if I have a search page defined and I didn't put in
      > > http:// then maybe it should do a search on whatever it
      > > couldn't find.
      >
      > Test: User enters "gmail" in address bar and presses Enter.
      > Result: System navigates to the user-configured search page,
      > with "gmail" as the search parameter.
      >
      > > Or if I didn't put in http:// and I don't have a search
      > > page what should it do?
      >
      > What do you want it to do? It could display an error message page.
      > It could use a default search engine. Once you know what result you
      > want, you can test for it.
      >
      > > And since I am using FF it also had to find a list of all
      > > the pages I have visited recently that match the things
      > > I type in.
      >
      > A test might need to clear the "recently visited" list, then visit a
      > few sites, then test this logic.
      >
    • Rick Mugridge
      ... I believe that it s not the scale of the test that should determine who writes it. It s instead whether the tests expresses stuff that s part of the
      Message 36 of 36 , Nov 8, 2008
      • 0 Attachment
        J. B. Rainsberger wrote:
        > I suggest programmers focus on isolated object tests and testers focus
        > on integration and end-to-end tests. If they do that, then they'll
        > come together pretty well at some point.
        > ----

        I believe that it's not the scale of the test that should determine who
        writes it.

        It's instead whether the tests expresses stuff that's part of the
        problem space or part of the solution space. Of course, where that
        boundary sits is critically dependent on the project and who is
        involved. And it changes as the problem, and solution, are better
        understood.

        And there can be several layers, with a solution space at one level
        being a problem space at another. So, for example, I'm happy to use
        storytests for specifying the technical details of communication with
        another system that is managed by another team. And I'm happy to have
        some storytests that mock out that other system so that we can use
        additive "specification"/testing rather than multiplicative across the
        systems. As always, we still need some end-to-end to ensure it's all
        wired together correctly and that failure modes across them are managed
        correctly.

        So I find the usual distinctions between unit tests and end-to-end tests
        and X, Y, Z tests to be unhelpful. As it's too late and too hard to
        refactor the terminology, I try (unsuccessfully) to avoid it.

        I prefer Brian Marick's distinction between customer-facing and
        programmer-facing tests.

        Cheers, Rick
      Your message has been successfully submitted and would be delivered to recipients shortly.