Loading ...
Sorry, an error occurred while loading the content.

RE: Why preserve TDD tests anyway (was RE: [XP] Testing Leftover)

Expand Messages
  • Lior Fridman
    I want to focus on the point several people have made about the ability to test all the classes by using only public interfaces.I agree!If the behavior of
    Message 1 of 12 , Jun 1, 2004
    • 0 Attachment
      I want to focus on the point several people have made about the ability
      to test all the classes by using only public interfaces.

      I agree!

      If the behavior of all the interfaces is correct than I do not care what
      the class does inside.
      However if I go back to my case it all started out as you said
      I wrote down the tests I thought needed to test the class behavior (via
      using the interfaces only)
      And than I started coding. At some point ive got all the tests working
      and I said "ok now lets
      make the code cleaner" I don't want to use the refacturing term cause I
      don't really think I truly used any refacturing mechanisms,
      but the intention was the same.

      So after doing this for a while ive got to the point when one of my
      tests broke
      The reason was that I made a stupid mistake in changing the registration
      method a mistake that was only caught later on by a test to the
      monitoring part.

      Now it took me a while to find this out so I wanted to make sure that
      this wont happen again
      And here is were this thread came into being. I didn't have a way to
      check it.

      So ive added a function that searched the list of the registered
      components to check if after I register a component it is there.
      Now I needed to place this function. So ok it's a basic getter/finder
      function lets add it to the class.
      But wait no one outside the testing actually need it so I don't want to
      add a chunk of unused code into the class so lets put it in the test
      class and make the test class a friend, It actually stayed this way for
      a couple of weeks until I wanted to publicate the whole thing and I
      noticed I just did something very bad. Left testing code in the
      operational code.
      And this is the story of this thread.

      The morale of all this is (I think)
      1) yes I caught the problem by only testing interfaces.
      2) I still want the ability to test private methods/access private
      members.
      3) this ability will help me fix the problem faster (and not find
      problem that were not found by the interfaces tests)

      Lior


      Lior



      -----Original Message-----
      From: Michael Feathers [mailto:mfeathers@...]
      Sent: Monday, May 31, 2004 9:33 PM
      To: Steven Gordon
      Subject: Re: Why preserve TDD tests anyway (was RE: [XP] Testing
      Leftover)

      SG> I believe the underlying point of this thread is the question "Why
      SG> should we bother preserving the unit tests we develop under TDD
      SG> anyway?". After all, if anything breaks in the system, one or more
      acceptance tests should break, so why not just throw the unit tests away
      after they have guided our design and proved that our code works as we
      designed it to work!

      SG> We SHOULD preserve the unit tests we develop under TDD because of 2
      basic reasons:
      SG> 1. When refactoring later, the more granular our regression test
      SG> suite, the easier it is to identify the place where the
      code broke.
      SG> 2. To document our design in a way that can be verified by simply
      executing the documentation.

      Agreed.

      SG> Disposing of our unit tests after design is to lose valuable
      information. This is true of the unit tests of methods that were later
      made private, as well as the unit tests of public methods.

      Agreed, except there is no time called "after design."

      SG> In Java, reflection would be the cleanest approach to preserving
      SG> unit tests for private methods without warping your production code.

      SG> In C++, I would consider using an IFDEF around the "private:" in the
      .h files of the production classes. You could also break unit test
      cases for classes into one for the public methods and one for the
      private methods and use the same IFDEF to include/disclude those test
      cases from your CppUnit test suites.

      I agree except that.. the fact that we want to test a private method is
      giving us valuable design information too. The test is telling us that
      the method should be public. If not on this class, then on another.

      Classes that can not be tested through their public interface are not
      designed well. We can always design classes in a way that makes them
      directly testable.

      Michael Feathers
      www.objectmentor.com



      To Post a message, send it to: extremeprogramming@...

      To Unsubscribe, send a blank message to:
      extremeprogramming-unsubscribe@...

      ad-free courtesy of objectmentor.com
      Yahoo! Groups Links








      The information contained in this message is proprietary of Amdocs,
      protected from disclosure, and may be privileged.
      The information is intended to be conveyed only to the designated recipient(s)
      of the message. If the reader of this message is not the intended recipient,
      you are hereby notified that any dissemination, use, distribution or copying of
      this communication is strictly prohibited and may be unlawful.
      If you have received this communication in error, please notify us immediately
      by replying to the message and deleting it from your computer.
      Thank you.
    • Lior Fridman
      Shamefully yes. But hey im new to this whole XP/TDD/refacturing thingIt takes some time to digest ;)Lior ... Sent: Tuesday, June 01, 2004 10:29 AM To:
      Message 2 of 12 , Jun 1, 2004
      • 0 Attachment
        Shamefully yes.
        But hey im new to this whole XP/TDD/refacturing thing

        It takes some time to digest ;)

        Lior


        -----Original Message-----
        From: lasse.koskela@... [mailto:lasse.koskela@...]
        Sent: Tuesday, June 01, 2004 10:29 AM
        To: extremeprogramming@yahoogroups.com
        Subject: VS: Why preserve TDD tests anyway (was RE: [XP] Testing
        Leftover)

        Lior Fridman:
        > I don't want to use the refacturing term cause I don't really think I
        > truly used any refacturing mechanisms, but the intention was the same.

        So you /did refactoring/, but didn't use (known or documented)
        /refactorings/ ;-)

        - Lasse -





        This message is for the designated recipient only and may contain
        privileged, proprietary, or otherwise private information. If you have
        received it in error, please notify the sender immediately and delete
        the original. Any other use of the email by you is prohibited.


        [Non-text portions of this message have been removed]



        To Post a message, send it to: extremeprogramming@...

        To Unsubscribe, send a blank message to:
        extremeprogramming-unsubscribe@...

        ad-free courtesy of objectmentor.com
        Yahoo! Groups Links








        The information contained in this message is proprietary of Amdocs,
        protected from disclosure, and may be privileged.
        The information is intended to be conveyed only to the designated recipient(s)
        of the message. If the reader of this message is not the intended recipient,
        you are hereby notified that any dissemination, use, distribution or copying of
        this communication is strictly prohibited and may be unlawful.
        If you have received this communication in error, please notify us immediately
        by replying to the message and deleting it from your computer.
        Thank you.
      • Ron Jeffries
        ... An observation in passing: I usually just go one test at a time in TDD. Sometimes I ll keep a list, but often I do not. ... Observation: It might be fun to
        Message 3 of 12 , Jun 1, 2004
        • 0 Attachment
          On Tuesday, June 1, 2004, at 3:03:58 AM, Lior Fridman wrote:

          > I want to focus on the point several people have made about the ability
          > to test all the classes by using only public interfaces.

          > I agree!

          > If the behavior of all the interfaces is correct than I do not care what
          > the class does inside.
          > However if I go back to my case it all started out as you said
          > I wrote down the tests I thought needed to test the class behavior (via
          > using the interfaces only)

          An observation in passing: I usually just go one test at a time in TDD.
          Sometimes I'll keep a list, but often I do not.

          > And than I started coding. At some point ive got all the tests working
          > and I said "ok now lets
          > make the code cleaner" I don't want to use the refacturing term cause I
          > don't really think I truly used any refacturing mechanisms,
          > but the intention was the same.

          Observation: It might be fun to keep track of the things we did to make
          the code cleaner, and to figure out which, if any, were named refactorings.
          Probably a lot were. In the course of learning which ones were, I bet we'd
          also learn some new ones, and learn better ways of doing the ones we did.

          > So after doing this for a while ive got to the point when one of my
          > tests broke
          > The reason was that I made a stupid mistake in changing the registration
          > method a mistake that was only caught later on by a test to the
          > monitoring part.

          > Now it took me a while to find this out so I wanted to make sure that
          > this wont happen again
          > And here is were this thread came into being. I didn't have a way to
          > check it.

          Yes. IMO, the "right" discipline is always to improve the testing when a
          defect is found other than in a direct way.

          > So ive added a function that searched the list of the registered
          > components to check if after I register a component it is there.
          > Now I needed to place this function. So ok it's a basic getter/finder
          > function lets add it to the class.
          > But wait no one outside the testing actually need it so I don't want to
          > add a chunk of unused code into the class so lets put it in the test
          > class and make the test class a friend, It actually stayed this way for
          > a couple of weeks until I wanted to publicate the whole thing and I
          > noticed I just did something very bad. Left testing code in the
          > operational code.
          > And this is the story of this thread.

          Well, I wouldn't worry, personally. However, the story sounds like a mock
          registration object of some kind could in fact find the bug.

          > The morale of all this is (I think)
          > 1) yes I caught the problem by only testing interfaces.
          > 2) I still want the ability to test private methods/access private
          > members.
          > 3) this ability will help me fix the problem faster (and not find
          > problem that were not found by the interfaces tests)

          If I want to test private methods, I test them. I do it using any number of
          techniques; most commonly the C# protection mode that allows anyone in the
          same namespace to access the item in question.

          In this case, it sounds like a complex registration sequence failed, and
          testing of complex sequences is often done with mocks, so I'd consider that
          alternative.

          Ron Jeffries
          www.XProgramming.com
          It's easier to act your way into a new way of thinking
          than to think your way into a new way of acting. --Millard Fuller
        • Michael Feathers
          Hello Lior, Tuesday, June 1, 2004, 3:03:58 AM, you wrote: LF I want to focus on the point several people have made about the ability LF to test all the
          Message 4 of 12 , Jun 1, 2004
          • 0 Attachment
            Hello Lior,

            Tuesday, June 1, 2004, 3:03:58 AM, you wrote:


            LF> I want to focus on the point several people have made about the ability
            LF> to test all the classes by using only public interfaces.

            LF> I agree!

            LF> If the behavior of all the interfaces is correct than I do not care what
            LF> the class does inside.
            LF> However if I go back to my case it all started out as you said
            LF> I wrote down the tests I thought needed to test the class behavior (via
            LF> using the interfaces only)
            LF> And than I started coding. At some point ive got all the tests working
            LF> and I said "ok now lets
            LF> make the code cleaner" I don't want to use the refacturing term cause I
            LF> don't really think I truly used any refacturing mechanisms,
            LF> but the intention was the same.

            LF> So after doing this for a while ive got to the point when one of my
            LF> tests broke
            LF> The reason was that I made a stupid mistake in changing the registration
            LF> method a mistake that was only caught later on by a test to the
            LF> monitoring part.

            LF> Now it took me a while to find this out so I wanted to make sure that
            LF> this wont happen again
            LF> And here is were this thread came into being. I didn't have a way to
            LF> check it.

            LF> So ive added a function that searched the list of the registered
            LF> components to check if after I register a component it is there.
            LF> Now I needed to place this function. So ok it's a basic getter/finder
            LF> function lets add it to the class.
            LF> But wait no one outside the testing actually need it so I don't want to
            LF> add a chunk of unused code into the class so lets put it in the test
            LF> class and make the test class a friend, It actually stayed this way for
            LF> a couple of weeks until I wanted to publicate the whole thing and I
            LF> noticed I just did something very bad. Left testing code in the
            LF> operational code.
            LF> And this is the story of this thread.

            Lior, I know this has been mentioned before but I didn't see you
            respond. If you have a class A which registers components with B why
            not have a mock B which allows you to search and use it under test?
            That way, you don't have this finder in your production code and you
            are able to see A's impact on B.

            Michael Feathers
            www.objectmentor.com
          • Lior Fridman
            since I see a lot of confusion here I think id better explain again (and better) the example I gaveIn short we have 2 classes: A,B B is the class I want to
            Message 5 of 12 , Jun 1, 2004
            • 0 Attachment
              since I see a lot of confusion here I think id better explain again
              (and better) the example I gave

              In short we have 2 classes: A,B
              B is the class I want to check. One of its interfaces is a register
              function (which is used by A to register himself to B)
              The registering mechanism (to keep the example simple) add the object
              which is passed to him (of type B) into his private data structure (in
              our case a list).

              The test I want to compose goes like this:
              1) B.Register(*A)
              2) check if A is in B.list.

              Problem I don't have a B.isThere(A) function or a B.getList(). since I
              don't really need it (other than for the testing part)

              So replacing B wouldn't help too much since I want to check B.
              I can create a mock A but than again it wont help me too much since I
              would still need access to private members of B and were back at square
              1.

              Hope this makes things clearer

              Lior

              -----Original Message-----
              From: Michael Feathers [mailto:mfeathers@...]
              Sent: Tuesday, June 01, 2004 2:42 PM
              To: Lior Fridman
              Subject: Re[2]: Why preserve TDD tests anyway (was RE: [XP] Testing
              Leftover)

              Hello Lior,

              Tuesday, June 1, 2004, 3:03:58 AM, you wrote:


              LF> I want to focus on the point several people have made about the
              LF> ability to test all the classes by using only public interfaces.

              LF> I agree!

              LF> If the behavior of all the interfaces is correct than I do not care
              LF> what the class does inside.
              LF> However if I go back to my case it all started out as you said I
              LF> wrote down the tests I thought needed to test the class behavior
              LF> (via using the interfaces only) And than I started coding. At some
              LF> point ive got all the tests working and I said "ok now lets make the

              LF> code cleaner" I don't want to use the refacturing term cause I
              LF> don't really think I truly used any refacturing mechanisms, but the
              LF> intention was the same.

              LF> So after doing this for a while ive got to the point when one of my
              LF> tests broke The reason was that I made a stupid mistake in changing
              LF> the registration method a mistake that was only caught later on by a

              LF> test to the monitoring part.

              LF> Now it took me a while to find this out so I wanted to make sure
              LF> that this wont happen again And here is were this thread came into
              LF> being. I didn't have a way to check it.

              LF> So ive added a function that searched the list of the registered
              LF> components to check if after I register a component it is there.
              LF> Now I needed to place this function. So ok it's a basic
              LF> getter/finder function lets add it to the class.
              LF> But wait no one outside the testing actually need it so I don't want

              LF> to add a chunk of unused code into the class so lets put it in the
              LF> test class and make the test class a friend, It actually stayed this

              LF> way for a couple of weeks until I wanted to publicate the whole
              LF> thing and I noticed I just did something very bad. Left testing code

              LF> in the operational code.
              LF> And this is the story of this thread.

              Lior, I know this has been mentioned before but I didn't see you
              respond. If you have a class A which registers components with B why
              not have a mock B which allows you to search and use it under test?
              That way, you don't have this finder in your production code and you are
              able to see A's impact on B.

              Michael Feathers
              www.objectmentor.com



              To Post a message, send it to: extremeprogramming@...

              To Unsubscribe, send a blank message to:
              extremeprogramming-unsubscribe@...

              ad-free courtesy of objectmentor.com
              Yahoo! Groups Links








              The information contained in this message is proprietary of Amdocs,
              protected from disclosure, and may be privileged.
              The information is intended to be conveyed only to the designated recipient(s)
              of the message. If the reader of this message is not the intended recipient,
              you are hereby notified that any dissemination, use, distribution or copying of
              this communication is strictly prohibited and may be unlawful.
              If you have received this communication in error, please notify us immediately
              by replying to the message and deleting it from your computer.
              Thank you.
            • Ron Jeffries
              ... What if you subclass B to BTest, and give it an isThere() function? Ron Jeffries www.XProgramming.com Master your instrument, master the music, and then
              Message 6 of 12 , Jun 2, 2004
              • 0 Attachment
                On Wednesday, June 2, 2004, at 2:35:01 AM, Lior Fridman wrote:

                > Problem I don't have a B.isThere(A) function or a B.getList(). since I
                > don't really need it (other than for the testing part)

                > So replacing B wouldn't help too much since I want to check B.
                > I can create a mock A but than again it wont help me too much since I
                > would still need access to private members of B and were back at square
                > 1.

                > Hope this makes things clearer

                What if you subclass B to BTest, and give it an isThere() function?

                Ron Jeffries
                www.XProgramming.com
                Master your instrument, master the music,
                and then forget all that *!xy!@ and just play. -- Charlie Parker
              • Michael Feathers
                ... RJ What if you subclass B to BTest, and give it an isThere() function? Yes, a very direct way of finding out. Another question is whether B s behavior
                Message 7 of 12 , Jun 2, 2004
                • 0 Attachment
                  RJ> On Wednesday, June 2, 2004, at 2:35:01 AM, Lior Fridman wrote:

                  >> Problem I don't have a B.isThere(A) function or a B.getList(). since I
                  >> don't really need it (other than for the testing part)

                  >> So replacing B wouldn't help too much since I want to check B.
                  >> I can create a mock A but than again it wont help me too much since I
                  >> would still need access to private members of B and were back at square
                  >> 1.

                  >> Hope this makes things clearer

                  RJ> What if you subclass B to BTest, and give it an isThere() function?

                  Yes, a very direct way of finding out. Another question is whether
                  B's behavior will be any different if it has an A in it. If it will
                  be, you can test B's behavior to detect the presence of A. If B's
                  behavior isn't any different at all, that is kind of spooky.


                  Michael Feathers
                  www.objectmentor.com
                • Steve Bate
                  ... Lior, What behavior from B do you expect if A was properly registered? Does B emit events based on it s monitoring activities? How does B monitor the
                  Message 8 of 12 , Jun 2, 2004
                  • 0 Attachment
                    > From: Lior Fridman [mailto:lior.fridman@...]
                    >...
                    > The test I want to compose goes like this:
                    > 1) B.Register(*A)
                    > 2) check if A is in B.list.
                    >
                    > Problem I don't have a B.isThere(A) function or a B.getList(). since I
                    > don't really need it (other than for the testing part)

                    Lior,

                    What behavior from B do you expect if A was properly registered?
                    Does B emit events based on it's monitoring activities? How does
                    B monitor the registered objects? Does it have it's own thread or
                    is it triggered by a method call or incoming event or ???.

                    Some methods only create private changes in a tested object. By
                    definition, those methods can't be tested alone without making
                    that private information accessible in some way. The problem
                    with that approach is that it creates unnecessary coupling
                    between the test and the component's internal implementation
                    details. I'm assuming the private side effects of the
                    registration method lead to some publicly visible behavior
                    during B's monitoring activities. One way to test the
                    registration is to test that subsequent expected behavior.

                    For example...

                    0) Set up A so it will trigger a monitoring event
                    1) B.Register(*A)
                    2) tell B to monitor
                    3) assert that the A-related monitoring event was emitted

                    This tests both the registration and the monitoring rather than
                    separating them into their own tests. There's no dependencies
                    on B's private data structures so those can vary without breaking
                    the test as long as the externally visible behavior does not
                    change.

                    We haven't had the issues some people are describing with diagnosing
                    problems identified by this form of test. Our team believes this is
                    a better approach than adding test-only functionality in production
                    code or breaking implementation encapsulation through private field
                    or method access. Our experience is that these techniques increase
                    test brittleness.

                    Steve
                  • Keith Ray
                    Look, B is just a container with extra functionality. Containers have putters and getters. It makes sense for B to have a public isThere function. ... --
                    Message 9 of 12 , Jun 2, 2004
                    • 0 Attachment
                      Look, B is just a "container" with extra functionality. Containers have
                      putters and getters. It makes sense for B to have a public "isThere"
                      function.

                      On Jun 1, 2004, at 11:35 PM, Lior Fridman wrote:

                      >
                      > since I see a lot of confusion here I think id better explain again
                      > (and better) the example I gave
                      >
                      > In short we have 2 classes: A,B
                      > B is the class I want to check. One of its interfaces is a register
                      > function (which is used by A to register himself to B)
                      > The registering mechanism (to keep the example simple) add the object
                      > which is passed to him (of type B) into his private data structure (in
                      > our case a list).
                      >
                      > The test I want to compose goes like this:
                      > 1) B.Register(*A)
                      > 2) check if A is in B.list.
                      >
                      > Problem I don't have a B.isThere(A) function or a B.getList(). since I
                      > don't really need it (other than for the testing part)
                      >
                      > So replacing B wouldn't help too much since I want to check B.
                      > I can create a mock A but than again it wont help me too much since I
                      > would still need access to private members of B and were back at square
                      > 1.
                      >
                      > Hope this makes things clearer
                      >
                      > Lior
                      >
                      > -----Original Message-----
                      > From: Michael Feathers [mailto:mfeathers@...]
                      >
                      > Sent: Tuesday, June 01, 2004 2:42 PM
                      > To: Lior Fridman
                      > Subject: Re[2]: Why preserve TDD tests anyway (was RE: [XP] Testing
                      > Leftover)
                      >
                      > Hello Lior,
                      >
                      > Tuesday, June 1, 2004, 3:03:58 AM, you wrote:
                      >
                      >
                      > LF> I want to focus on the point several people have made about the
                      >
                      > LF> ability to test all the classes by using only public interfaces.
                      >
                      > LF> I agree!
                      >
                      > LF> If the behavior of all the interfaces is correct than I do not care
                      >
                      > LF> what the class does inside.
                      > LF> However if I go back to my case it all started out as you said I
                      >
                      > LF> wrote down the tests I thought needed to test the class behavior
                      >
                      > LF> (via using the interfaces only) And than I started coding. At some
                      >
                      > LF> point ive got all the tests working and I said "ok now lets make
                      > the
                      >
                      > LF> code cleaner" I don't want to use the refacturing term cause I
                      >
                      > LF> don't really think I truly used any refacturing mechanisms, but the
                      >
                      > LF> intention was the same.
                      >
                      > LF> So after doing this for a while ive got to the point when one of my
                      >
                      > LF> tests broke The reason was that I made a stupid mistake in changing
                      >
                      > LF> the registration method a mistake that was only caught later on by
                      > a
                      >
                      > LF> test to the monitoring part.
                      >
                      > LF> Now it took me a while to find this out so I wanted to make sure
                      >
                      > LF> that this wont happen again And here is were this thread came into
                      >
                      > LF> being. I didn't have a way to check it.
                      >
                      > LF> So ive added a function that searched the list of the registered
                      >
                      > LF> components to check if after I register a component it is there.
                      > LF> Now I needed to place this function. So ok it's a basic
                      >
                      > LF> getter/finder function lets add it to the class.
                      > LF> But wait no one outside the testing actually need it so I don't
                      > want
                      >
                      > LF> to add a chunk of unused code into the class so lets put it in the
                      >
                      > LF> test class and make the test class a friend, It actually stayed
                      > this
                      >
                      > LF> way for a couple of weeks until I wanted to publicate the whole
                      >
                      > LF> thing and I noticed I just did something very bad. Left testing
                      > code
                      >
                      > LF> in the operational code.
                      > LF> And this is the story of this thread.
                      >
                      > Lior, I know this has been mentioned before but I didn't see you
                      > respond. If you have a class A which registers components with B why
                      > not have a mock B which allows you to search and use it under test?
                      > That way, you don't have this finder in your production code and you
                      > are
                      > able to see A's impact on B.
                      >
                      > Michael Feathers
                      > www.objectmentor.com
                      >
                      >
                      >
                      > To Post a message, send it to: extremeprogramming@...
                      >
                      > To Unsubscribe, send a blank message to:
                      > extremeprogramming-unsubscribe@...
                      >
                      > ad-free courtesy of objectmentor.com
                      > Yahoo! Groups Links
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      > The information contained in this message is proprietary of Amdocs,
                      > protected from disclosure, and may be privileged.
                      > The information is intended to be conveyed only to the designated
                      > recipient(s)
                      > of the message. If the reader of this message is not the intended
                      > recipient,
                      > you are hereby notified that any dissemination, use, distribution or
                      > copying of
                      >
                      > this communication is strictly prohibited and may be unlawful.
                      >
                      > If you have received this communication in error, please notify us
                      > immediately
                      > by replying to the message and deleting it from your computer.
                      > Thank you.
                      >
                      >
                      > To Post a message, send it to: extremeprogramming@...
                      >
                      > To Unsubscribe, send a blank message to:
                      > extremeprogramming-unsubscribe@...
                      >
                      > ad-free courtesy of objectmentor.com
                      > Yahoo! Groups Links
                      >
                      >
                      >
                      >
                      >
                      >
                      --
                      C. Keith Ray
                      <http://homepage.mac.com/keithray/blog/index.html>
                      <http://homepage.mac.com/keithray/xpminifaq.html>
                      <http://homepage.mac.com/keithray/resume2.html>
                    Your message has been successfully submitted and would be delivered to recipients shortly.