Loading ...
Sorry, an error occurred while loading the content.

RE: [XP] Re: Test execution speed

Expand Messages
  • Steve Bate
    ... Hi Dominic, We ve had quite different experiences and results. Although our current team is mostly in the seasoned OO programmer category, I ve seen the
    Message 1 of 260 , Aug 2, 2004
    • 0 Attachment
      > From: Dominic Williams [mailto:yg_xp@...]
      > However, TDD is also a design technique. I have
      > observed that mocking or stubbing make it possible to
      > unit test fragile, rigid, highly coupled designs. I
      > feel therefore that unit testing without mocks or stubs
      > has a more positive influence on the design. This is
      > important when working with teams in which everyone may
      > not be a long-seasoned OO programmer.

      Hi Dominic,

      We've had quite different experiences and results. Although
      our current team is mostly in the seasoned OO programmer
      category, I've seen the same unit test techniques work well
      with teams of less experienced or skilled programmers (after
      some initial coaching). It's possible that some of the
      differences in our results might be due to different

      You are using the term "stubbing" here in addition to mocking.
      How do you define those terms? Our mocks have essentially
      no behavior. A mocked method returns exactly what it is
      explicitly told to return. They never generate any results.
      For our functional tests, we sometimes use what we call
      simulators. A simulator does have behavior and might
      generate results. For example, our application is an
      equities and derivative trading system. The functional tests
      use a simulator that mimics the behavior of the external
      equity and derivative exchanges.

      We have had no problem (that I remember) with increased
      coupling due to mock-based unit testing. Given that each
      class is dependent only on abstractions (interfaces) has the
      opposite effect. Could you give some examples of how mocking
      leads to highly coupled designs?

      > Another concern I have with respect to your approach is
      > that although the application's classes have been
      > decoupled, the unit tests have become coupled with the
      > code's implementation.
      > An important aim of object-oriented design is hiding
      > the implementation details. I have found TDD to be very
      > useful in helping programmers think about objects from
      > an external, client perspective. Tests coded this way
      > are independent of the implementation of the object
      > under test. Stubbing or mocking the collaborators of
      > the object under test makes the tests dependent on that
      > implementation. On long-running projects, I have
      > observed that this has two drawbacks:

      You are correct. A unit test is testing a concrete class
      implementation and is exposed to that concrete class's
      collaborators. Even so, a unit test can be written from a
      black box (vs. white or gray box) perspective so that the
      exposure is only to the collaborators and not to other
      implementation details. The benefit of using mocks for
      the collaborators is that the unit test is only exposed
      to the /direct/ collaborators.

      Suppose you have a class A that depends on B and C, and B
      depends on D and E, and C depends on F and G, and so on. In
      a unit test, we'd mock B and C. The test would have to set
      up and tear down those two mocks. If we don't use mocks, we
      have to set up and tear down B,C,D,E,F,G and all the other
      indirect dependencies. The test is not only dependent on
      the implementation of the class under test but also on the
      implementations of all indirectly dependent classes.

      One side effect of mocking is that it's a pain to have too
      many direct dependencies from a class. This naturally triggers
      us to look for higher level, meaningful abstractions that
      simply the design and testing. This is one force that leads
      to a Repository abstraction and associated mocks rather than
      depending directly on JDBC or file access code. Setting up
      the relatively simple mock Repository calls is easy compared
      to setting up mock JDBC calls. It's these types of
      considerations that make me think that our mock-based testing
      leads to a more decoupled design rather than less. Even the
      less experienced programmers quickly learn than decoupling
      and creating good abstractions will make their testing work

      When doing TDD, I find it valuable to be able to focus on
      the class I'm designing and the abstractions it depends upon
      rather than having to design the implementation of all
      dependent, concrete classes at the same time. Again,
      decoupling happens almost automatically in this context

      > 1) the code/test base is less flexible: each change
      > requires more work.

      Let's say you have a concrete class X that, indirectly
      (possibly through many levels of indirection), is used
      by almost every class in the system. The internal
      implementation of X is modified in a way that requires it
      to be set up slightly differently (e.g., a constructor
      argument is added). The setup method for every test case
      that tests an object indirectly related to X must now be
      modified. If X mocks were used, only the X test case would
      be modified. All other tests are not dependent on the
      concrete implementation of X. I realize a team could
      mitigate this issue by using an "object mother" rather
      than mocks. I haven't used that approach so I don't know
      how much work is usually involved in modifying the
      related object mother methods in a scenario like this one.

      > 2) refactoring is less safe: it is almost always
      > necessary to change both code and unit tests at the
      > same time.

      Again, it depends on the type of refactoring, but I agree
      this is often the case. How do you avoid modifying tests
      that depend directly and indirectly on refactored concrete

    • Ilja Preuss
      ... Yes, but I thought that we were talking about a test that was wrong. Not sure wether that matters, though... Cheers, Ilja
      Message 260 of 260 , Aug 18, 2004
      • 0 Attachment
        Adrian Howard wrote:
        > On 17 Aug 2004, at 12:22, Ilja Preuss wrote:
        > [snip]
        >> It's certainly the case that without pairing/reviews I am more
        >> likely to
        >> *miss* tests - but I don't think that I get more *wrong* tests that
        >> cancel out with wrong implementation...
        > I think it could happen over time.
        > - Lack of pairing might mean I miss duplication so a bit
        > of business logic gets into foo and bar.
        > - My acceptance test for the business logic only uses foo.
        > - Later I change bar incorrectly, but the foo test still passes.
        > False-pass for that bit of business logic.

        Yes, but I thought that we were talking about a test that was wrong. Not
        sure wether that matters, though...

        Cheers, Ilja
      Your message has been successfully submitted and would be delivered to recipients shortly.