Loading ...
Sorry, an error occurred while loading the content.

Re: Why preserve TDD tests anyway (was RE: [XP] Testing Leftover)

Expand Messages
  • Ron Jeffries
    ... An observation in passing: I usually just go one test at a time in TDD. Sometimes I ll keep a list, but often I do not. ... Observation: It might be fun to
    Message 1 of 12 , Jun 1, 2004
    • 0 Attachment
      On Tuesday, June 1, 2004, at 3:03:58 AM, Lior Fridman wrote:

      > I want to focus on the point several people have made about the ability
      > to test all the classes by using only public interfaces.

      > I agree!

      > If the behavior of all the interfaces is correct than I do not care what
      > the class does inside.
      > However if I go back to my case it all started out as you said
      > I wrote down the tests I thought needed to test the class behavior (via
      > using the interfaces only)

      An observation in passing: I usually just go one test at a time in TDD.
      Sometimes I'll keep a list, but often I do not.

      > And than I started coding. At some point ive got all the tests working
      > and I said "ok now lets
      > make the code cleaner" I don't want to use the refacturing term cause I
      > don't really think I truly used any refacturing mechanisms,
      > but the intention was the same.

      Observation: It might be fun to keep track of the things we did to make
      the code cleaner, and to figure out which, if any, were named refactorings.
      Probably a lot were. In the course of learning which ones were, I bet we'd
      also learn some new ones, and learn better ways of doing the ones we did.

      > So after doing this for a while ive got to the point when one of my
      > tests broke
      > The reason was that I made a stupid mistake in changing the registration
      > method a mistake that was only caught later on by a test to the
      > monitoring part.

      > Now it took me a while to find this out so I wanted to make sure that
      > this wont happen again
      > And here is were this thread came into being. I didn't have a way to
      > check it.

      Yes. IMO, the "right" discipline is always to improve the testing when a
      defect is found other than in a direct way.

      > So ive added a function that searched the list of the registered
      > components to check if after I register a component it is there.
      > Now I needed to place this function. So ok it's a basic getter/finder
      > function lets add it to the class.
      > But wait no one outside the testing actually need it so I don't want to
      > add a chunk of unused code into the class so lets put it in the test
      > class and make the test class a friend, It actually stayed this way for
      > a couple of weeks until I wanted to publicate the whole thing and I
      > noticed I just did something very bad. Left testing code in the
      > operational code.
      > And this is the story of this thread.

      Well, I wouldn't worry, personally. However, the story sounds like a mock
      registration object of some kind could in fact find the bug.

      > The morale of all this is (I think)
      > 1) yes I caught the problem by only testing interfaces.
      > 2) I still want the ability to test private methods/access private
      > members.
      > 3) this ability will help me fix the problem faster (and not find
      > problem that were not found by the interfaces tests)

      If I want to test private methods, I test them. I do it using any number of
      techniques; most commonly the C# protection mode that allows anyone in the
      same namespace to access the item in question.

      In this case, it sounds like a complex registration sequence failed, and
      testing of complex sequences is often done with mocks, so I'd consider that
      alternative.

      Ron Jeffries
      www.XProgramming.com
      It's easier to act your way into a new way of thinking
      than to think your way into a new way of acting. --Millard Fuller
    • Michael Feathers
      Hello Lior, Tuesday, June 1, 2004, 3:03:58 AM, you wrote: LF I want to focus on the point several people have made about the ability LF to test all the
      Message 2 of 12 , Jun 1, 2004
      • 0 Attachment
        Hello Lior,

        Tuesday, June 1, 2004, 3:03:58 AM, you wrote:


        LF> I want to focus on the point several people have made about the ability
        LF> to test all the classes by using only public interfaces.

        LF> I agree!

        LF> If the behavior of all the interfaces is correct than I do not care what
        LF> the class does inside.
        LF> However if I go back to my case it all started out as you said
        LF> I wrote down the tests I thought needed to test the class behavior (via
        LF> using the interfaces only)
        LF> And than I started coding. At some point ive got all the tests working
        LF> and I said "ok now lets
        LF> make the code cleaner" I don't want to use the refacturing term cause I
        LF> don't really think I truly used any refacturing mechanisms,
        LF> but the intention was the same.

        LF> So after doing this for a while ive got to the point when one of my
        LF> tests broke
        LF> The reason was that I made a stupid mistake in changing the registration
        LF> method a mistake that was only caught later on by a test to the
        LF> monitoring part.

        LF> Now it took me a while to find this out so I wanted to make sure that
        LF> this wont happen again
        LF> And here is were this thread came into being. I didn't have a way to
        LF> check it.

        LF> So ive added a function that searched the list of the registered
        LF> components to check if after I register a component it is there.
        LF> Now I needed to place this function. So ok it's a basic getter/finder
        LF> function lets add it to the class.
        LF> But wait no one outside the testing actually need it so I don't want to
        LF> add a chunk of unused code into the class so lets put it in the test
        LF> class and make the test class a friend, It actually stayed this way for
        LF> a couple of weeks until I wanted to publicate the whole thing and I
        LF> noticed I just did something very bad. Left testing code in the
        LF> operational code.
        LF> And this is the story of this thread.

        Lior, I know this has been mentioned before but I didn't see you
        respond. If you have a class A which registers components with B why
        not have a mock B which allows you to search and use it under test?
        That way, you don't have this finder in your production code and you
        are able to see A's impact on B.

        Michael Feathers
        www.objectmentor.com
      • Lior Fridman
        since I see a lot of confusion here I think id better explain again (and better) the example I gaveIn short we have 2 classes: A,B B is the class I want to
        Message 3 of 12 , Jun 1, 2004
        • 0 Attachment
          since I see a lot of confusion here I think id better explain again
          (and better) the example I gave

          In short we have 2 classes: A,B
          B is the class I want to check. One of its interfaces is a register
          function (which is used by A to register himself to B)
          The registering mechanism (to keep the example simple) add the object
          which is passed to him (of type B) into his private data structure (in
          our case a list).

          The test I want to compose goes like this:
          1) B.Register(*A)
          2) check if A is in B.list.

          Problem I don't have a B.isThere(A) function or a B.getList(). since I
          don't really need it (other than for the testing part)

          So replacing B wouldn't help too much since I want to check B.
          I can create a mock A but than again it wont help me too much since I
          would still need access to private members of B and were back at square
          1.

          Hope this makes things clearer

          Lior

          -----Original Message-----
          From: Michael Feathers [mailto:mfeathers@...]
          Sent: Tuesday, June 01, 2004 2:42 PM
          To: Lior Fridman
          Subject: Re[2]: Why preserve TDD tests anyway (was RE: [XP] Testing
          Leftover)

          Hello Lior,

          Tuesday, June 1, 2004, 3:03:58 AM, you wrote:


          LF> I want to focus on the point several people have made about the
          LF> ability to test all the classes by using only public interfaces.

          LF> I agree!

          LF> If the behavior of all the interfaces is correct than I do not care
          LF> what the class does inside.
          LF> However if I go back to my case it all started out as you said I
          LF> wrote down the tests I thought needed to test the class behavior
          LF> (via using the interfaces only) And than I started coding. At some
          LF> point ive got all the tests working and I said "ok now lets make the

          LF> code cleaner" I don't want to use the refacturing term cause I
          LF> don't really think I truly used any refacturing mechanisms, but the
          LF> intention was the same.

          LF> So after doing this for a while ive got to the point when one of my
          LF> tests broke The reason was that I made a stupid mistake in changing
          LF> the registration method a mistake that was only caught later on by a

          LF> test to the monitoring part.

          LF> Now it took me a while to find this out so I wanted to make sure
          LF> that this wont happen again And here is were this thread came into
          LF> being. I didn't have a way to check it.

          LF> So ive added a function that searched the list of the registered
          LF> components to check if after I register a component it is there.
          LF> Now I needed to place this function. So ok it's a basic
          LF> getter/finder function lets add it to the class.
          LF> But wait no one outside the testing actually need it so I don't want

          LF> to add a chunk of unused code into the class so lets put it in the
          LF> test class and make the test class a friend, It actually stayed this

          LF> way for a couple of weeks until I wanted to publicate the whole
          LF> thing and I noticed I just did something very bad. Left testing code

          LF> in the operational code.
          LF> And this is the story of this thread.

          Lior, I know this has been mentioned before but I didn't see you
          respond. If you have a class A which registers components with B why
          not have a mock B which allows you to search and use it under test?
          That way, you don't have this finder in your production code and you are
          able to see A's impact on B.

          Michael Feathers
          www.objectmentor.com



          To Post a message, send it to: extremeprogramming@...

          To Unsubscribe, send a blank message to:
          extremeprogramming-unsubscribe@...

          ad-free courtesy of objectmentor.com
          Yahoo! Groups Links








          The information contained in this message is proprietary of Amdocs,
          protected from disclosure, and may be privileged.
          The information is intended to be conveyed only to the designated recipient(s)
          of the message. If the reader of this message is not the intended recipient,
          you are hereby notified that any dissemination, use, distribution or copying of
          this communication is strictly prohibited and may be unlawful.
          If you have received this communication in error, please notify us immediately
          by replying to the message and deleting it from your computer.
          Thank you.
        • Ron Jeffries
          ... What if you subclass B to BTest, and give it an isThere() function? Ron Jeffries www.XProgramming.com Master your instrument, master the music, and then
          Message 4 of 12 , Jun 2, 2004
          • 0 Attachment
            On Wednesday, June 2, 2004, at 2:35:01 AM, Lior Fridman wrote:

            > Problem I don't have a B.isThere(A) function or a B.getList(). since I
            > don't really need it (other than for the testing part)

            > So replacing B wouldn't help too much since I want to check B.
            > I can create a mock A but than again it wont help me too much since I
            > would still need access to private members of B and were back at square
            > 1.

            > Hope this makes things clearer

            What if you subclass B to BTest, and give it an isThere() function?

            Ron Jeffries
            www.XProgramming.com
            Master your instrument, master the music,
            and then forget all that *!xy!@ and just play. -- Charlie Parker
          • Michael Feathers
            ... RJ What if you subclass B to BTest, and give it an isThere() function? Yes, a very direct way of finding out. Another question is whether B s behavior
            Message 5 of 12 , Jun 2, 2004
            • 0 Attachment
              RJ> On Wednesday, June 2, 2004, at 2:35:01 AM, Lior Fridman wrote:

              >> Problem I don't have a B.isThere(A) function or a B.getList(). since I
              >> don't really need it (other than for the testing part)

              >> So replacing B wouldn't help too much since I want to check B.
              >> I can create a mock A but than again it wont help me too much since I
              >> would still need access to private members of B and were back at square
              >> 1.

              >> Hope this makes things clearer

              RJ> What if you subclass B to BTest, and give it an isThere() function?

              Yes, a very direct way of finding out. Another question is whether
              B's behavior will be any different if it has an A in it. If it will
              be, you can test B's behavior to detect the presence of A. If B's
              behavior isn't any different at all, that is kind of spooky.


              Michael Feathers
              www.objectmentor.com
            • Steve Bate
              ... Lior, What behavior from B do you expect if A was properly registered? Does B emit events based on it s monitoring activities? How does B monitor the
              Message 6 of 12 , Jun 2, 2004
              • 0 Attachment
                > From: Lior Fridman [mailto:lior.fridman@...]
                >...
                > The test I want to compose goes like this:
                > 1) B.Register(*A)
                > 2) check if A is in B.list.
                >
                > Problem I don't have a B.isThere(A) function or a B.getList(). since I
                > don't really need it (other than for the testing part)

                Lior,

                What behavior from B do you expect if A was properly registered?
                Does B emit events based on it's monitoring activities? How does
                B monitor the registered objects? Does it have it's own thread or
                is it triggered by a method call or incoming event or ???.

                Some methods only create private changes in a tested object. By
                definition, those methods can't be tested alone without making
                that private information accessible in some way. The problem
                with that approach is that it creates unnecessary coupling
                between the test and the component's internal implementation
                details. I'm assuming the private side effects of the
                registration method lead to some publicly visible behavior
                during B's monitoring activities. One way to test the
                registration is to test that subsequent expected behavior.

                For example...

                0) Set up A so it will trigger a monitoring event
                1) B.Register(*A)
                2) tell B to monitor
                3) assert that the A-related monitoring event was emitted

                This tests both the registration and the monitoring rather than
                separating them into their own tests. There's no dependencies
                on B's private data structures so those can vary without breaking
                the test as long as the externally visible behavior does not
                change.

                We haven't had the issues some people are describing with diagnosing
                problems identified by this form of test. Our team believes this is
                a better approach than adding test-only functionality in production
                code or breaking implementation encapsulation through private field
                or method access. Our experience is that these techniques increase
                test brittleness.

                Steve
              • Keith Ray
                Look, B is just a container with extra functionality. Containers have putters and getters. It makes sense for B to have a public isThere function. ... --
                Message 7 of 12 , Jun 2, 2004
                • 0 Attachment
                  Look, B is just a "container" with extra functionality. Containers have
                  putters and getters. It makes sense for B to have a public "isThere"
                  function.

                  On Jun 1, 2004, at 11:35 PM, Lior Fridman wrote:

                  >
                  > since I see a lot of confusion here I think id better explain again
                  > (and better) the example I gave
                  >
                  > In short we have 2 classes: A,B
                  > B is the class I want to check. One of its interfaces is a register
                  > function (which is used by A to register himself to B)
                  > The registering mechanism (to keep the example simple) add the object
                  > which is passed to him (of type B) into his private data structure (in
                  > our case a list).
                  >
                  > The test I want to compose goes like this:
                  > 1) B.Register(*A)
                  > 2) check if A is in B.list.
                  >
                  > Problem I don't have a B.isThere(A) function or a B.getList(). since I
                  > don't really need it (other than for the testing part)
                  >
                  > So replacing B wouldn't help too much since I want to check B.
                  > I can create a mock A but than again it wont help me too much since I
                  > would still need access to private members of B and were back at square
                  > 1.
                  >
                  > Hope this makes things clearer
                  >
                  > Lior
                  >
                  > -----Original Message-----
                  > From: Michael Feathers [mailto:mfeathers@...]
                  >
                  > Sent: Tuesday, June 01, 2004 2:42 PM
                  > To: Lior Fridman
                  > Subject: Re[2]: Why preserve TDD tests anyway (was RE: [XP] Testing
                  > Leftover)
                  >
                  > Hello Lior,
                  >
                  > Tuesday, June 1, 2004, 3:03:58 AM, you wrote:
                  >
                  >
                  > LF> I want to focus on the point several people have made about the
                  >
                  > LF> ability to test all the classes by using only public interfaces.
                  >
                  > LF> I agree!
                  >
                  > LF> If the behavior of all the interfaces is correct than I do not care
                  >
                  > LF> what the class does inside.
                  > LF> However if I go back to my case it all started out as you said I
                  >
                  > LF> wrote down the tests I thought needed to test the class behavior
                  >
                  > LF> (via using the interfaces only) And than I started coding. At some
                  >
                  > LF> point ive got all the tests working and I said "ok now lets make
                  > the
                  >
                  > LF> code cleaner" I don't want to use the refacturing term cause I
                  >
                  > LF> don't really think I truly used any refacturing mechanisms, but the
                  >
                  > LF> intention was the same.
                  >
                  > LF> So after doing this for a while ive got to the point when one of my
                  >
                  > LF> tests broke The reason was that I made a stupid mistake in changing
                  >
                  > LF> the registration method a mistake that was only caught later on by
                  > a
                  >
                  > LF> test to the monitoring part.
                  >
                  > LF> Now it took me a while to find this out so I wanted to make sure
                  >
                  > LF> that this wont happen again And here is were this thread came into
                  >
                  > LF> being. I didn't have a way to check it.
                  >
                  > LF> So ive added a function that searched the list of the registered
                  >
                  > LF> components to check if after I register a component it is there.
                  > LF> Now I needed to place this function. So ok it's a basic
                  >
                  > LF> getter/finder function lets add it to the class.
                  > LF> But wait no one outside the testing actually need it so I don't
                  > want
                  >
                  > LF> to add a chunk of unused code into the class so lets put it in the
                  >
                  > LF> test class and make the test class a friend, It actually stayed
                  > this
                  >
                  > LF> way for a couple of weeks until I wanted to publicate the whole
                  >
                  > LF> thing and I noticed I just did something very bad. Left testing
                  > code
                  >
                  > LF> in the operational code.
                  > LF> And this is the story of this thread.
                  >
                  > Lior, I know this has been mentioned before but I didn't see you
                  > respond. If you have a class A which registers components with B why
                  > not have a mock B which allows you to search and use it under test?
                  > That way, you don't have this finder in your production code and you
                  > are
                  > able to see A's impact on B.
                  >
                  > Michael Feathers
                  > www.objectmentor.com
                  >
                  >
                  >
                  > To Post a message, send it to: extremeprogramming@...
                  >
                  > To Unsubscribe, send a blank message to:
                  > extremeprogramming-unsubscribe@...
                  >
                  > ad-free courtesy of objectmentor.com
                  > Yahoo! Groups Links
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  > The information contained in this message is proprietary of Amdocs,
                  > protected from disclosure, and may be privileged.
                  > The information is intended to be conveyed only to the designated
                  > recipient(s)
                  > of the message. If the reader of this message is not the intended
                  > recipient,
                  > you are hereby notified that any dissemination, use, distribution or
                  > copying of
                  >
                  > this communication is strictly prohibited and may be unlawful.
                  >
                  > If you have received this communication in error, please notify us
                  > immediately
                  > by replying to the message and deleting it from your computer.
                  > Thank you.
                  >
                  >
                  > To Post a message, send it to: extremeprogramming@...
                  >
                  > To Unsubscribe, send a blank message to:
                  > extremeprogramming-unsubscribe@...
                  >
                  > ad-free courtesy of objectmentor.com
                  > Yahoo! Groups Links
                  >
                  >
                  >
                  >
                  >
                  >
                  --
                  C. Keith Ray
                  <http://homepage.mac.com/keithray/blog/index.html>
                  <http://homepage.mac.com/keithray/xpminifaq.html>
                  <http://homepage.mac.com/keithray/resume2.html>
                Your message has been successfully submitted and would be delivered to recipients shortly.