Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Why do I test private methods?

Expand Messages
  • wconrad@sprintmail.com
    ... Yes, there is value in testing a private method. Consider whether there would be even more value in making the method a public member of another class. A
    Message 1 of 20 , Sep 27, 2000
    • 0 Attachment
      On Wed, Sep 27, 2000 at 07:25:56PM -0500, Joi Ellis wrote:
      > Don't tell me testing non-public methods has no value. If I had
      > followed my usual practice of testing new methods, I wouldn't have
      > wasted two days this week over such a simple bug.

      Yes, there is value in testing a private method.

      Consider whether there would be even more value in making the method a
      public member of another class. A private method which can be tested
      easily can usually be be moved easily (if it's easy to test, it's
      probably not strongly coupled to its class). This may improve the
      code as well as expose the method for testing.

      Example on request.

      Wayne Conrad
    • Joi Ellis
      I ve just spent two days trying to get my JNI library to properly talk to our hardware over the network. I had a public method, openUdpConnection(), and a
      Message 2 of 20 , Sep 27, 2000
      • 0 Attachment
        I've just spent two days trying to get my JNI library to properly talk
        to our hardware over the network.

        I had a public method, openUdpConnection(), and a non-public helper
        method int = convertIpStringToInt(String addy). I didn't have a test
        on the convertIpStringToInt method, but that turned out to be where the
        bug was. It was a simple one-liner, and I had it wrong.

        As soon as I added a test to this non-public method I realized I had
        written the expression incorrectly.

        In two days of work I hadn't figured this out, but once I unit-tested
        that method, I got the whole thing running.

        Don't tell me testing non-public methods has no value. If I had
        followed my usual practice of testing new methods, I wouldn't have
        wasted two days this week over such a simple bug.


        --
        Joi Ellis Software Engineer
        Aravox Technologies joi@..., gyles19@...

        No matter what we think of Linux versus FreeBSD, etc., the one thing I
        really like about Linux is that it has Microsoft worried. Anything
        that kicks a monopoly in the pants has got to be good for something.
        - Chris Johnson
      • Ron Jeffries
        ... Perfect example. A private method may be easy to test in isolation, but the problems it causes may be hard to see in the tests for the public methods that
        Message 3 of 20 , Sep 27, 2000
        • 0 Attachment
          At 07:25 PM 9/27/00 -0500, Joi Ellis wrote:
          >I had a public method, openUdpConnection(), and a non-public helper
          >method int = convertIpStringToInt(String addy). I didn't have a test
          >on the convertIpStringToInt method, but that turned out to be where the
          >bug was. It was a simple one-liner, and I had it wrong.
          >
          >As soon as I added a test to this non-public method I realized I had
          >written the expression incorrectly.
          >
          >In two days of work I hadn't figured this out, but once I unit-tested
          >that method, I got the whole thing running.
          >
          >Don't tell me testing non-public methods has no value. If I had
          >followed my usual practice of testing new methods, I wouldn't have
          >wasted two days this week over such a simple bug.

          Perfect example. A private method may be easy to test in isolation, but the
          problems it causes may be hard to see in the tests for the public methods
          that use it. IMO there is no reason to avoid testing private methods if
          they could break - and as in your example - even if they "couldn't".

          Thanks!

          Ronald E Jeffries
          http://www.XProgramming.com
          http://www.objectmentor.com
        • Jonathan Rasmusson
          ... I couldn t agree with you more. In my experience, just as many things can go wrong in a private method as they can in public methods. I see no difference
          Message 4 of 20 , Sep 27, 2000
          • 0 Attachment
            --- In extremeprogramming@egroups.com, Joi Ellis <joi@a...> wrote:
            >
            > Don't tell me testing non-public methods has no value. If I had
            > followed my usual practice of testing new methods, I wouldn't have
            > wasted two days this week over such a simple bug.

            I couldn't agree with you more. In my experience, just as many
            things can go wrong in a private method as they can in public
            methods. I see no difference in writing unit tests for each.
            If there is any chance of it breaking, let yourself rest easy
            at night by writing a dumb test to prove it.

            You lose nothing and gain a lot more confidence.

            Jonathan
          • Arrizza, John
            ... The private method you mention is a special case of a private method. Most private methods are used internally to the class, i.e. there is *no* outside
            Message 5 of 20 , Sep 28, 2000
            • 0 Attachment
              > -----Original Message-----
              > From: Joi Ellis [mailto:joi@...]
              > Don't tell me testing non-public methods has no value. If I had
              > followed my usual practice of testing new methods, I wouldn't have
              > wasted two days this week over such a simple bug.

              The private method you mention is a special case of a private method. Most
              private methods are used internally to the class, i.e. there is *no* outside
              connection.

              The method you describe is different in that there is an outside connection,
              in this case, literally. I had something similar in a private method that
              looked at the system clock. The method failed after 28.4 days because the
              clock rolled over to 0.

              So I made a change and waited 28.5 days to test it... (kidding). I wrapped
              the clock() call in a static function and then used a #if around the static
              function so that I could get to it from my UT.

              John
            • Joi Ellis
              ... I don t follow you. This non-public method is used only by the package its its, there are no outside callers to it, that s why it s not public. The only
              Message 6 of 20 , Sep 28, 2000
              • 0 Attachment
                "Arrizza, John" wrote:
                >
                > > -----Original Message-----
                > > From: Joi Ellis [mailto:joi@...]
                > > Don't tell me testing non-public methods has no value. If I had
                > > followed my usual practice of testing new methods, I wouldn't have
                > > wasted two days this week over such a simple bug.
                >
                > The private method you mention is a special case of a private method. Most
                > private methods are used internally to the class, i.e. there is *no* outside
                > connection.

                I don't follow you. This non-public method is used only by the package its
                its, there are no outside callers to it, that's why it's not public. The
                only reason it's not on the same class is that I have deliberately split
                all methods which are part of the JNI API onto a singleton class that tracks
                the system library they're calling. That way I can replace the whole library
                with a Mock if I need to by changing the single method that loads the
                singleton. If I wasn't going to do that I could have put all of it on one
                class object.

                > The method you describe is different in that there is an outside connection,
                > in this case, literally. I had something similar in a private method that
                > looked at the system clock. The method failed after 28.4 days because the
                > clock rolled over to 0.

                The calling method is attemping to build an API call which will in turn
                open a socket. The caller itself isn't doing anything but being a wrapper,
                converting fields to the formats the API wants (all ints and byte[] things.)

                Someone else suggested I move the method to the class that's using it. It
                already is on the calling class. Another suggested making it public. Why?
                No one else is using it (yet) and it doesn't need to be public to be tested.
                None of these would have changed the bug in it and the initial lack of a test
                for it.

                As for why I didn't catch it in the caller's unit test: I was trying to make
                the caller's very first unit test run. I only have one datapoint it that
                case, because all of the test methods on the test class were failing, some
                with slightly different errors than others which only confused me further.

                What was the bug? I was taking 4 numeric strings (the dotted quad tokens)
                and shifting them into the bytes of a 4-byte int. The problem was I was
                shifting nybbles, not bytes. Sigh. And, since I made the same logical
                mistake in writing the test for it, it puzzled me for several more minutes
                after the method passed its first unit test and the caller continued to
                report failures.

                --
                Joi Ellis Software Engineer
                Aravox Technologies joi@..., gyles19@...

                No matter what we think of Linux versus FreeBSD, etc., the one thing I
                really like about Linux is that it has Microsoft worried. Anything
                that kicks a monopoly in the pants has got to be good for something.
                - Chris Johnson
              • Mark Wilden
                ... From: Ron Jeffries ... Yes, but doing so does not assure that it is being used correctly by the public methods. ... methods ...
                Message 7 of 20 , Sep 28, 2000
                • 0 Attachment
                  ----- Original Message -----
                  From: "Ron Jeffries" <ronjeffries@...>
                  > A private method may be easy to test in isolation

                  Yes, but doing so does not assure that it is being used correctly by the
                  public methods.

                  > but the problems it causes may be hard to see in the tests for the public
                  methods
                  > that use it.

                  This is the part I don't understand.

                  Let's say you have a block of three lines of code in a public method. You
                  write a test for the public method, and the test passes. Is it conceivable
                  that those three lines of code cause problems that are hard to see? Yes, I
                  suppose so. So what do you do? You improve your public method test until
                  those problems are not hard to see, but are tested for and exposed.

                  Now, how does the situation change if you move those three lines of code
                  into a private method? Don't you still need to include those "hard to see"
                  problems in the public method test? If so, are you going to test for them
                  again in the private method test?
                • Eric Hodges
                  1. How did you know there was a bug to look for? Did you unit tests of public methods reveal that fact? 2. You say private in the subject line, but
                  Message 8 of 20 , Sep 28, 2000
                  • 0 Attachment
                    1. How did you know there was a bug to look for? Did you unit tests of
                    public methods reveal that fact?
                    2. You say "private" in the subject line, but "non-public" everywhere else.
                    Was this a package scope method?

                    If so, package scope methods are not private. I see nothing wrong or
                    difficult in testing package scope methods. Testing truly private methods
                    is more difficult and IMHO less useful.


                    > -----Original Message-----
                    > From: joi@... [mailto:joi@...]On Behalf Of Joi
                    > Ellis
                    > Sent: Wednesday, September 27, 2000 7:26 PM
                    > To: extremeprogramming@...
                    > Subject: [XP] Why do I test private methods?
                    >
                    >
                    > I've just spent two days trying to get my JNI library to properly talk
                    > to our hardware over the network.
                    >
                    > I had a public method, openUdpConnection(), and a non-public helper
                    > method int = convertIpStringToInt(String addy). I didn't have a test
                    > on the convertIpStringToInt method, but that turned out to be where the
                    > bug was. It was a simple one-liner, and I had it wrong.
                    >
                    > As soon as I added a test to this non-public method I realized I had
                    > written the expression incorrectly.
                    >
                    > In two days of work I hadn't figured this out, but once I unit-tested
                    > that method, I got the whole thing running.
                    >
                    > Don't tell me testing non-public methods has no value. If I had
                    > followed my usual practice of testing new methods, I wouldn't have
                    > wasted two days this week over such a simple bug.
                    >
                    >
                    > --
                    > Joi Ellis Software Engineer
                    > Aravox Technologies joi@..., gyles19@...
                    >
                    > No matter what we think of Linux versus FreeBSD, etc., the one thing I
                    > really like about Linux is that it has Microsoft worried. Anything
                    > that kicks a monopoly in the pants has got to be good for something.
                    > - Chris Johnson
                    >
                    > To Post a message, send it to: extremeprogramming@...
                    >
                    > To Unsubscribe, send a blank message to:
                    > extremeprogramming-unsubscribe@...
                    >
                    > Ad-free courtesy of objectmentor.com
                    >
                  • Phil Goodwin
                    ... That wasn t a test -- it was a diagnostic. The difference is subtle, but important: you need tests to tell you that something is wrong; you need
                    Message 9 of 20 , Sep 28, 2000
                    • 0 Attachment
                      At 11:26 PM 9/27/00 -0400, Ron Jeffries wrote:
                      >At 07:25 PM 9/27/00 -0500, Joi Ellis wrote:
                      > >I had a public method, openUdpConnection(), and a non-public helper
                      > >method int = convertIpStringToInt(String addy). I didn't have a test
                      > >on the convertIpStringToInt method, but that turned out to be where the
                      > >bug was. It was a simple one-liner, and I had it wrong.
                      > >
                      > >As soon as I added a test to this non-public method I realized I had
                      > >written the expression incorrectly.
                      > >
                      > >In two days of work I hadn't figured this out, but once I unit-tested
                      > >that method, I got the whole thing running.
                      > >
                      > >Don't tell me testing non-public methods has no value. If I had
                      > >followed my usual practice of testing new methods, I wouldn't have
                      > >wasted two days this week over such a simple bug.
                      >
                      >Perfect example. A private method may be easy to test in isolation, but the
                      >problems it causes may be hard to see in the tests for the public methods
                      >that use it. IMO there is no reason to avoid testing private methods if
                      >they could break - and as in your example - even if they "couldn't".
                      >
                      >Thanks!

                      That wasn't a test -- it was a diagnostic. The difference is subtle, but
                      important: you need tests to tell you that something is wrong; you need
                      diagnostics to tell you what it is.

                      If the the line of code in question had been inlined to the public method
                      instead of extracted to a private method the bug would have been just as
                      hard to find.

                      If you had run your tests more frequently you'd have seen that the failure
                      occurred right after you wrote that one line private method.

                      Diagnostics are fine as throw away tools. Tests are a little more durable.
                      That's because a test is an abstract statement about what a system should
                      do -- they can be run against many systems and used to separate the systems
                      that meet the requirements from those that don't. Diagnostics only apply to
                      one program. Once you change the program the diagnostic doesn't apply any
                      more and can be thrown away. If you plan to change your program often it
                      may not be a good idea to invest much time in writing diagnostics.
                      Fortunately there are a number of general purpose diagnostic tools
                      available. They are called debuggers and many of them are quite good. When
                      one of my tests fail I look at the code and if I don't spot the problem
                      immediately I re-run the program under the debugger and step through the
                      offending code. Since all my tests exercise only a small amount of code I
                      never have to look too far to find the problem. I have no problem looking
                      at private data or running private methods this way of course.

                      I think that any time a non-public part of a class is tested the test is
                      really a diagnostic and time is being wasted (you can always run the thing
                      in the debugger while you are developing it). Since tests are long lasting
                      and debugging sessions aren't, I think that it's wise to use tests to check
                      out the stable (public) parts of the system and debugging sessions to check
                      out the less stable (private) parts of the system.

                      ------------------------------------------------------------------------------------------------------------
                      Phil Goodwin, Java Software, Sun Microsystems, 408-517-6951, or x66951

                      "The significant problems we face cannot be solved at the same level of
                      thinking we were at when we created them." -- Albert Einstein
                    • Arrizza, John
                      ... not ... I ve reread you re original post. My apologies for misreading it. (However, my point still remains about private methods that wrap or use external
                      Message 10 of 20 , Sep 28, 2000
                      • 0 Attachment
                        > -----Original Message-----
                        > From: Joi Ellis [mailto:joi@...]
                        > I don't follow you. This non-public method is used only by
                        > the package its in, there are no outside callers to it, that's why it's
                        not
                        > public.

                        I've reread you're original post. My apologies for misreading it.

                        (However, my point still remains about private methods that wrap or use
                        external components that change at will (volatile?). Unit tests should be
                        written in those cases even though the method is not part of the public
                        interface.)

                        > Someone else suggested I move the method to the class that's
                        > using it. It already is on the calling class. Another suggested making
                        it
                        > public. Why?
                        No need to make it public. The method is not part of the public interface,
                        don't put it there.

                        Perhaps the simplest thing to do here is to say that Unit Testing does not
                        mean 0 bugs. It just means many bugs are cleaned up early in the development
                        cycle.

                        Another way to look at it: say 3 classes are 95% clean, their interaction is
                        .95*.95*.95 = 86% clean. If you use UTs and they are now 99.5% clean then
                        the 3 classes overall are 98.5% clean.

                        If 99.5% clean is good, is 100% clean better? No. That *might* be a waste of
                        time and energy. Test everything you can ("that could possibly break ...")
                        with an eye towards your efficacy as a coder.

                        John
                      • John D. Mitchell
                        ... [...] ... Excellent point! Sometimes they are one and the same, sometimes they are wildly different. FWIW, I find that this distinction is one of the nice
                        Message 11 of 20 , Sep 28, 2000
                        • 0 Attachment
                          >>>>> "Phil" == Phil Goodwin <phil.goodwin@...> writes:
                          [...]
                          > That wasn't a test -- it was a diagnostic. The difference is subtle, but
                          > important: you need tests to tell you that something is wrong; you need
                          > diagnostics to tell you what it is.

                          Excellent point!

                          Sometimes they are one and the same, sometimes they are wildly different.
                          FWIW, I find that this distinction is one of the nice ways to frame
                          thinking about the distinctions between unit testing from things like
                          assertions and DBC.

                          [...]
                          > I think that any time a non-public part of a class is tested the test is
                          > really a diagnostic and time is being wasted (you can always run the
                          > thing in the debugger while you are developing it). Since tests are long
                          > lasting and debugging sessions aren't, I think that it's wise to use
                          > tests to check out the stable (public) parts of the system and debugging
                          > sessions to check out the less stable (private) parts of the system.

                          IMHO, you're making this (sounde to me to be) way too cut and dried. See
                          my previous messages in the parent of this thread for more on the
                          public/private thing.

                          One of the big points about XP is that this stuff needs to be repeatable
                          and automated. Purely relying on the debugger (and developer for execution
                          observation/inspection) is, IMHO, just as silly as trying to completely
                          automate absolutely everything. IME, if an issue comes up where it's
                          really worth the time/effort in cranking up the debugger and stepping
                          through the code then it's critical to update/add new unit tests.

                          You use of the concept of "stability" seems interesting but I think that's
                          a backwards conclusion. I.e., it is, if anything, *more* important to nail
                          down those unstable areas than the "stable" areas because it's those
                          unstable areas that are going to hose you because they will blindside you.

                          Take care,
                          John

                          --
                          John D. Mitchell
                          jGuru.com -- Today is a Great Day to Code!
                          NoWebPatents.org -- Stop Patent Stupidity!
                        • Ron Jeffries
                          ... suppose the method says doOneBigThing(); doAnotherBigThing(); return aFinalBigThing(); where all those are private methods. We test the method and lo! we
                          Message 12 of 20 , Sep 28, 2000
                          • 0 Attachment
                            At 09:26 AM 9/28/00 -0700, Mark Wilden wrote:
                            >This is the part I don't understand.
                            >
                            >Let's say you have a block of three lines of code in a public method. You
                            >write a test for the public method, and the test passes. Is it conceivable
                            >that those three lines of code cause problems that are hard to see? Yes, I
                            >suppose so. So what do you do? You improve your public method test until
                            >those problems are not hard to see, but are tested for and exposed.
                            >
                            >Now, how does the situation change if you move those three lines of code
                            >into a private method? Don't you still need to include those "hard to see"
                            >problems in the public method test? If so, are you going to test for them
                            >again in the private method test?

                            suppose the method says
                            doOneBigThing();
                            doAnotherBigThing();
                            return aFinalBigThing();

                            where all those are private methods. We test the method and lo! we get the
                            wrong answer. Now we have a debugging problem, and we don't even know
                            whether one of the first two methods has left the object in an invalid
                            state, or whether aFinalBigThing computed the answer incorrectly.

                            Tests for the private methods doOneBigThing() and so on (perhaps even using
                            other private methods to rip the guts out of the object and look at them)
                            can identify which of the methods contains the bug. Debugging is reduced.
                            Debug time is wasted time. Testing a complex private method independently
                            can reduce debug time. Therefore testing a complex private method
                            independently can be a good thing.

                            Ronald E Jeffries
                            http://www.XProgramming.com
                            http://www.objectmentor.com
                          • Mark Wilden
                            ... From: Ron Jeffries ... OK, as long as one realizes that this says nothing about whether the software actually works. For example,
                            Message 13 of 20 , Sep 28, 2000
                            • 0 Attachment
                              ----- Original Message -----
                              From: "Ron Jeffries" <ronjeffries@...>
                              > Debugging is reduced.
                              > Debug time is wasted time. Testing a complex private method independently
                              > can reduce debug time. Therefore testing a complex private method
                              > independently can be a good thing.

                              OK, as long as one realizes that this says nothing about whether the
                              software actually works.

                              For example, you can test each of these methods in isolation and have them
                              pass with flying colors. But if you accidentally call them like this


                              > doAnotherBigThing();
                              > doOneBigThing();
                              > return aFinalBigThing();

                              your private method tests still pass, yet the software is (probably) wrong.

                              Using tests as debugging aids makes some sense. However, I just don't have
                              time to routinely write tests for private methods just in case I might need
                              to debug them one day. Maybe my work is easy enough so that debugging
                              doesn't take that much of my time.
                            • Phil Goodwin
                              ... I actually do think that it is cut and dried. Public members have a behavior that is expected of them by their clients. Tests can be used to define that
                              Message 14 of 20 , Sep 28, 2000
                              • 0 Attachment
                                At 01:00 PM 9/28/00 -0700, John D. Mitchell wrote:
                                > >>>>> "Phil" == Phil Goodwin <phil.goodwin@...> writes:
                                > > I think that any time a non-public part of a class is tested the test is
                                > > really a diagnostic and time is being wasted (you can always run the
                                > > thing in the debugger while you are developing it). Since tests are long
                                > > lasting and debugging sessions aren't, I think that it's wise to use
                                > > tests to check out the stable (public) parts of the system and debugging
                                > > sessions to check out the less stable (private) parts of the system.
                                >
                                >IMHO, you're making this (sounde to me to be) way too cut and dried. See
                                >my previous messages in the parent of this thread for more on the
                                >public/private thing.

                                I actually do think that it is cut and dried. Public members have a
                                behavior that is expected of them by their clients. Tests can be used to
                                define that behavior and demonstrate that a particular method exhibits it.
                                Private methods are, by definition, implementation details. Their behavior
                                can vary freely so long as the public methods don't break. Put another way:
                                if every public method has been proven to behave correctly then the class
                                has been proven to be bug free. Therefore, there is never a need to test
                                the private members of a class.

                                But people do it anyway. They don't want to stop. Why?

                                Because tests can be useful diagnostic tools. If you have private behavior
                                that is complex, stable and interesting enough to warrant testing then
                                that's a smell. That's code that's begging to be extracted into its own
                                class. If the code is simple or temporary or you already know that it works
                                (maybe you used to have a test for it that you deleted when you made it
                                private), then there is nothing that needs to be done. If that code ever
                                breaks then a debugger is likely to provide all the diagnostic power you
                                might need.

                                >One of the big points about XP is that this stuff needs to be repeatable
                                >and automated.

                                Tests need to be repeatable in order to demonstrate that the code still
                                works as expected. The only reason that diagnostics would need to be
                                repeatable is if the exact same bug keeps coming back.

                                >Purely relying on the debugger (and developer for execution
                                >observation/inspection) is, IMHO, just as silly as trying to completely
                                >automate absolutely everything.

                                I agree. I am not advocating that. I want tests to be repeatable and
                                diagnostic tools to be very flexible. Finding and removing a bug is a
                                one-shot event. It need not be repeatable.

                                >IME, if an issue comes up where it's
                                >really worth the time/effort in cranking up the debugger and stepping
                                >through the code then it's critical to update/add new unit tests.

                                My experience is that if a bug is that hard to find it's generally because
                                I've written too much code that isn't covered by a test. Then it's time to
                                step back and find a way to make things go in smaller, more numerous steps,
                                so I can test each one. I often find myself introducing an auxiliary class
                                at this time. I do not add tests for my private members, although I do, of
                                course, add tests for my new classes which often contain functionality that
                                used to be private. So I think that our experiences are similar although we
                                have chosen to respond to them differently.

                                >You use of the concept of "stability" seems interesting but I think that's
                                >a backwards conclusion. I.e., it is, if anything, *more* important to nail
                                >down those unstable areas than the "stable" areas because it's those
                                >unstable areas that are going to hose you because they will blindside you.

                                Well I think that sounds good, but that it won't stand up under scrutiny.
                                The key is that instability is actually a good thing in the right place and
                                in the right measure. If you "nail it down" then it becomes stable -- and
                                therefore inflexible. There is more than one way to produce correct
                                behavior. So long as you consistently find ways to provide provably correct
                                behavior for your public members you ought to be free to change their
                                implementation. This is the heart of the concept of encapsulation. Testing
                                private members breaks encapsulation both in the spirit and the letter of
                                the law, because if you write a test for a private member then you can't
                                change or remove it without breaking the test even if the external behavior
                                of the class remains correct. That is not the desired result of writing tests.




                                ------------------------------------------------------------------------------------------------------------
                                Phil Goodwin, Java Software, Sun Microsystems, 408-517-6951, or x66951

                                "The significant problems we face cannot be solved at the same level of
                                thinking we were at when we created them." -- Albert Einstein
                              • John D. Mitchell
                                ... [...] ... Sigh. You didn t read my messages in the parent thread of this thread. The point of contention is that it s *not* always possible (cost/effort
                                Message 15 of 20 , Sep 28, 2000
                                • 0 Attachment
                                  >>>>> "Phil" == Phil Goodwin <phil.goodwin@...> writes:
                                  >>>>>> At 01:00 PM 9/28/00 -0700, John D. Mitchell wrote:
                                  [...]
                                  >> IMHO, you're making this (sounde to me to be) way too cut and dried.
                                  >> See my previous messages in the parent of this thread for more on the
                                  >> public/private thing.

                                  > I actually do think that it is cut and dried. Public members have a
                                  > behavior that is expected of them by their clients. Tests can be used to
                                  > define that behavior and demonstrate that a particular method exhibits
                                  > it. Private methods are, by definition, implementation details. Their
                                  > behavior can vary freely so long as the public methods don't break. Put
                                  > another way: if every public method has been proven to behave correctly
                                  > then the class has been proven to be bug free. Therefore, there is never
                                  > a need to test the private members of a class.

                                  Sigh. You didn't read my messages in the parent thread of this thread.

                                  The point of contention is that it's *not* always possible (cost/effort
                                  effective, etc.) to write tests against *only* the public interface which
                                  does actually, fully, completely "prove" that the class is defect free.

                                  Again, I'm all for driving the tests purely based on the public interfaces
                                  as much as possible. It's just that your arguments are cut and dried and
                                  the facts are that software isn't always that cut and dried.

                                  [Since I seem to have to keep repeating myself on this very basic argument,
                                  I'll risk the wrath of hubris and point out that I have e.g., taught
                                  testing for Sun Labs so I do happen to have a wee bit of very applicable
                                  experience in this area. :-)]


                                  [...]
                                  >> One of the big points about XP is that this stuff needs to be repeatable
                                  >> and automated.

                                  > Tests need to be repeatable in order to demonstrate that the code still
                                  > works as expected. The only reason that diagnostics would need to be
                                  > repeatable is if the exact same bug keeps coming back.

                                  And if you don't have a test to drive that potential bug then you don't
                                  have a repeatable, automated process.

                                  To be extra clear, I do grok the distinction between diagnostics and (unit)
                                  tests and I think it's something that it is important for people to get.
                                  In this argument, I'm very clearly not confusing the two. Your implication
                                  that testing private methods is *only* for diagnostics is silly. If, for
                                  your code, it is actually the case that it's always only diagnostics then
                                  kudos.


                                  >> Purely relying on the debugger (and developer for execution
                                  >> observation/inspection) is, IMHO, just as silly as trying to completely
                                  >> automate absolutely everything.

                                  > I agree. I am not advocating that.

                                  Cool.

                                  > I want tests to be repeatable and diagnostic tools to be very
                                  > flexible. Finding and removing a bug is a one-shot event. It need not be
                                  > repeatable.

                                  If you don't have a repeatable, automated test to prove that it never
                                  reappears then something is missing. I.e., you might use the debugger to
                                  step through some code to spelunk the bug while I might write some
                                  diagnostic code to do the same. Either way, there must be a test written
                                  which clearly drives that bug.

                                  My guess is that our only fundamental disagreement on this point is what we
                                  each will mean by "clearly drives that bug". :-) You might be comfortable
                                  with just relying on the fact that an existing unit test failed and if it
                                  fails again then you (or your team members :-) will just re-spelunk the
                                  bug. I, on the other hand, want a bit more precision. So, within reason,
                                  it sounds like it's really just come down to a matter of style.


                                  >> IME, if an issue comes up where it's really worth the time/effort in
                                  >> cranking up the debugger and stepping through the code then it's
                                  >> critical to update/add new unit tests.

                                  > My experience is that if a bug is that hard to find it's generally
                                  > because I've written too much code that isn't covered by a test. Then
                                  > it's time to step back and find a way to make things go in smaller, more
                                  > numerous steps, so I can test each one. I often find myself introducing
                                  > an auxiliary class at this time. I do not add tests for my private
                                  > members, although I do, of course, add tests for my new classes which
                                  > often contain functionality that used to be private. So I think that our
                                  > experiences are similar although we have chosen to respond to them
                                  > differently.

                                  A point similar to one that I made in the parent thread of this thread. :-)

                                  I'll point out that I've made no argument about the level of difficulty of
                                  spelunking a bug. While I certainly agree that that's a factor, my
                                  perspective is that whether or not to test a given private method is often
                                  a more subtle matter of the "grip" available to drive the testing via the
                                  public interface. I.e., if the public interface gives you all of the grip
                                  you need to drive everything then rock on! If it doesn't then some
                                  white-box tests may be appropriate, refactoring may be appropriate, and
                                  even ignoring it may be appropriate.


                                  >> You use of the concept of "stability" seems interesting but I think
                                  >> that's a backwards conclusion. I.e., it is, if anything, *more*
                                  >> important to nail down those unstable areas than the "stable" areas
                                  >> because it's those unstable areas that are going to hose you because
                                  >> they will blindside you.

                                  > Well I think that sounds good, but that it won't stand up under scrutiny.
                                  > The key is that instability is actually a good thing in the right place
                                  > and in the right measure. If you "nail it down" then it becomes stable --
                                  > and therefore inflexible. There is more than one way to produce correct
                                  > behavior. So long as you consistently find ways to provide provably
                                  > correct behavior for your public members you ought to be free to change
                                  > their implementation. This is the heart of the concept of
                                  > encapsulation. Testing private members breaks encapsulation both in the
                                  > spirit and the letter of the law, because if you write a test for a
                                  > private member then you can't change or remove it without breaking the
                                  > test even if the external behavior of the class remains correct. That is
                                  > not the desired result of writing tests.

                                  Ah, sorry for the confusion. I'm not at all arguing that you should
                                  necessarily make all of an implementation "perfect". I'm arguing against
                                  your use of the "stability" judgement to refrain from adding regression
                                  *tests* which drive discovered flaws.

                                  W.r.t. the notion that testing private methods violates encapsulation.
                                  I'll go so far as to say that depends on how you're testing them. I'm an
                                  advocate of both white- and black-box testing. For example, in a Java
                                  context, I write tests specifically against the defined Java interface
                                  (i.e., black box) to cover the "contract" of the interface and then I write
                                  additional tests, as needed, for each class implementing that interface
                                  (which usually ends up as a mix of white- and black-box tests) which cover
                                  whatever makes that class unique from the pure contract. [If JavaSoft had
                                  done this sort of thing (including WTTF, etc.), just think how much better
                                  all of Java and the Java libraries would actually be. Sigh.]

                                  W.r.t. changing tests due to shifting behavior of the implementation. All
                                  I can think of is that you're making an incorrect presumption. I *never*
                                  change (nor advocate changing) the accessibility of a method merely for
                                  testing purposes. That's Just Plain Wrong(tm). It's quite easy in e.g.,
                                  Java to write tests for private methods without violating encapsulation or
                                  modifying the public interface of the class. Also, tests aren't ordained
                                  to be static and unchanging -- if the implementation changes then you
                                  should change the implementation specific tests appropriately; if the
                                  black-box tests break then either your changing the contract (and thus the
                                  appropriate black-box tests need to be modified/added) or your new
                                  implementation needs work.

                                  Take care,
                                  John

                                  --
                                  John D. Mitchell
                                  jGuru.com -- Today is a Great Day to Code!
                                  NoWebPatents.org -- Stop Patent Stupidity!
                                • Robert Watkins
                                  ... But then you ve failed to TETCPB, haven t you? The tests of the private methods assures you that your building blocks are sound. You then still have to
                                  Message 16 of 20 , Sep 28, 2000
                                  • 0 Attachment
                                    Mark Wilden writes:
                                    > For example, you can test each of these methods in isolation and have them
                                    > pass with flying colors. But if you accidentally call them like this
                                    >
                                    >
                                    > > doAnotherBigThing();
                                    > > doOneBigThing();
                                    > > return aFinalBigThing();
                                    >
                                    > your private method tests still pass, yet the software is
                                    > (probably) wrong.

                                    But then you've failed to TETCPB, haven't you? The tests of the private
                                    methods assures you that your building blocks are sound. You then still have
                                    to test to make sure you put them together correctly.

                                    If I build a bridge, I'm going to do things like test that the steel is
                                    strong enough for the use I'm putting it to. I'm also going to test that I
                                    put the bridge together correctly. (Then, because I'm a software engineer,
                                    and not a civil engineer, I'm never going to drive over it. ;)

                                    Robert.

                                    --
                                    /\\\ Robert Watkins | Email: robert.watkins@...
                                    ( ))) Software Engineer - Tech Dev | Ph: +61 7 3303 3432
                                    \/// MINCOM Ltd | Fax: +61 7 3303 3232
                                    Mincom

                                    This transmission is for the intended addressee only and is confidential
                                    information. If you have received this transmission in error, please
                                    delete it and notify the sender. The contents of this e-mail are the
                                    opinion of the writer only and are not endorsed by Mincom Limited unless
                                    expressly stated otherwise.
                                  • Robert Watkins
                                    ... Why can t you change or remove it? Because it will break the test? That sounds a bit weak to me, Phil. I highly doubt that every refactoring attempt will
                                    Message 17 of 20 , Sep 28, 2000
                                    • 0 Attachment
                                      Phil Goodwin writes:
                                      > Well I think that sounds good, but that it won't stand up under scrutiny.
                                      > The key is that instability is actually a good thing in the right
                                      > place and
                                      > in the right measure. If you "nail it down" then it becomes stable -- and
                                      > therefore inflexible. There is more than one way to produce correct
                                      > behavior. So long as you consistently find ways to provide
                                      > provably correct
                                      > behavior for your public members you ought to be free to change their
                                      > implementation. This is the heart of the concept of
                                      > encapsulation. Testing
                                      > private members breaks encapsulation both in the spirit and the letter of
                                      > the law, because if you write a test for a private member then you can't
                                      > change or remove it without breaking the test even if the
                                      > external behavior
                                      > of the class remains correct. That is not the desired result of
                                      > writing tests.

                                      Why can't you change or remove it? Because it will break the test? That
                                      sounds a bit weak to me, Phil.

                                      I highly doubt that every refactoring attempt will break absolutely 0 tests.
                                      The fact is that when you refactor beyond a certain point, some tests are
                                      going to break because the old behaviour is no longer in place. Those
                                      breaking tests then need to updated to reflect the new behaviour. What's
                                      wrong with that?

                                      Unit tests are for the developer. If it makes development faster to test
                                      private stuff, go ahead (and yes, it can make development faster... see my
                                      earlier post). Presumably, the time spent fixing tests that break when you
                                      refactor is made up by the efficiency gains.

                                      Now, if I saw someone doing acceptance tests that delved into the internals
                                      of the class, I'd take them around the back and beat them up.

                                      --
                                      /\\\ Robert Watkins | Email: robert.watkins@...
                                      ( ))) Software Engineer - Tech Dev | Ph: +61 7 3303 3432
                                      \/// MINCOM Ltd | Fax: +61 7 3303 3232
                                      Mincom

                                      This transmission is for the intended addressee only and is confidential
                                      information. If you have received this transmission in error, please
                                      delete it and notify the sender. The contents of this e-mail are the
                                      opinion of the writer only and are not endorsed by Mincom Limited unless
                                      expressly stated otherwise.
                                    • Eric Hodges
                                      ... From: Robert Watkins To: Sent: Thursday, September 28, 2000 7:52 PM Subject: RE: [XP] Why do
                                      Message 18 of 20 , Sep 28, 2000
                                      • 0 Attachment
                                        ----- Original Message -----
                                        From: "Robert Watkins" <robert.watkins@...>
                                        To: <extremeprogramming@...>
                                        Sent: Thursday, September 28, 2000 7:52 PM
                                        Subject: RE: [XP] Why do I test private methods?


                                        > Mark Wilden writes:
                                        > > For example, you can test each of these methods in isolation and have
                                        them
                                        > > pass with flying colors. But if you accidentally call them like this
                                        > >
                                        > >
                                        > > > doAnotherBigThing();
                                        > > > doOneBigThing();
                                        > > > return aFinalBigThing();
                                        > >
                                        > > your private method tests still pass, yet the software is
                                        > > (probably) wrong.
                                        >
                                        > But then you've failed to TETCPB, haven't you? The tests of the private
                                        > methods assures you that your building blocks are sound. You then still
                                        have
                                        > to test to make sure you put them together correctly.
                                        >
                                        > If I build a bridge, I'm going to do things like test that the steel is
                                        > strong enough for the use I'm putting it to. I'm also going to test that I
                                        > put the bridge together correctly. (Then, because I'm a software engineer,
                                        > and not a civil engineer, I'm never going to drive over it. ;)

                                        But then there's the Calvin & Hobbes story.

                                        Calvin and family are driving over a small bridge, with a sign that
                                        indicates the maximum weight it can hold. Calvin asks his dad how they find
                                        out the max weight. Calvin's dad says they build the bridge, then drive
                                        bigger and bigger trucks over it until it collapses, then build it again
                                        exactly the same way and post the weight of the next to the last truck.

                                        It's a joke, but that's exactly how a lot of software testing works. We
                                        don't have to do a lot of math and engineering to find out if something will
                                        break because we can test things without destroying them.
                                      • Phil Goodwin
                                        ... I went back and made sure that I got them all. I m pretty sure that I understand your point which is that private methods can t always be fully driven by
                                        Message 19 of 20 , Sep 29, 2000
                                        • 0 Attachment
                                          At 05:50 PM 9/28/00 -0700, John D. Mitchell wrote:

                                          >Sigh. You didn't read my messages in the parent thread of this thread.

                                          I went back and made sure that I got them all. I'm pretty sure that I
                                          understand your point which is that private methods can't always be fully
                                          driven by public methods. For instance if you have a private method that
                                          generates squares and a public method that only asks it to generate squares
                                          of even numbers then you can't test it to see if it generates squares of
                                          odd numbers.

                                          Hopefully I've got it right.

                                          I have two approaches to dealing with that. The first and most common, and
                                          the one that I expect that people (especially testing folks) won't like is
                                          -- I don't care. If the private method works well enough to make the public
                                          method work then it works well enough for me. The second approach I use is:
                                          actually I do care, it makes me nervous and I want to rid myself of the
                                          distraction. In that case I pull the whole thing out into another class and
                                          test it. Often things that are important enough to make me nervous are
                                          important enough to be their own class.

                                          >The point of contention is that it's *not* always possible (cost/effort
                                          >effective, etc.) to write tests against *only* the public interface which
                                          >does actually, fully, completely "prove" that the class is defect free.

                                          If a class satisfies its public contract then I consider it to be, by
                                          definition, defect free. I don't know of any counter examples and don't
                                          believe that there can be any. Maybe I'm being idealistic, but I'll need an
                                          actual example of something that is private, inextricably part of its
                                          class, and untestable from a public interface to convince me otherwise.

                                          >Again, I'm all for driving the tests purely based on the public interfaces
                                          >as much as possible. It's just that your arguments are cut and dried and
                                          >the facts are that software isn't always that cut and dried.

                                          Cut and dried arguments are enticing targets. They are almost always
                                          flawed. They are most powerful, however, when they manage to stand up to
                                          scrutiny. The argument that the public interface of a class together with
                                          its tests (including pre-condition assertions) is the definition of the
                                          class's contract is powerful and interesting enough that I feel compelled
                                          to continue defending it.

                                          >[Since I seem to have to keep repeating myself on this very basic argument,
                                          >I'll risk the wrath of hubris and point out that I have e.g., taught
                                          >testing for Sun Labs so I do happen to have a wee bit of very applicable
                                          >experience in this area. :-)]

                                          Some times the basics are the hardest things to get. Of course, even basic
                                          Newtonian physics, while dreadfully useful, is actually incorrect.

                                          > >> One of the big points about XP is that this stuff needs to be repeatable
                                          > >> and automated.
                                          >
                                          > > Tests need to be repeatable in order to demonstrate that the code still
                                          > > works as expected. The only reason that diagnostics would need to be
                                          > > repeatable is if the exact same bug keeps coming back.
                                          >
                                          >And if you don't have a test to drive that potential bug then you don't
                                          >have a repeatable, automated process.

                                          I accept the practice of adding unit tests that show a bug before fixing
                                          the bug. I am just asserting that all bugs can be demonstrated by unit
                                          testing only public members and that there are benefits to be reaped from
                                          doing so.

                                          The point I was trying to make above is that once you've changed the
                                          critical line or two that caused the bug it hardly makes sense to write a
                                          test whose only ability is to detect whether that line or two reverts back
                                          to its original state. Clearly that would be testing at too low a level.
                                          Similarly, and I think we are in agreement here, acceptance testing is
                                          testing at too high a level to be the only testing that ever gets done.
                                          There has to be a middle ground, a just right level, and I'm saying that
                                          the public interface of a class actually defines that level. I am actually
                                          surprised to have come to that conclusion. But I came by it honestly and I
                                          am delighted by it.

                                          >Your implication that testing private methods is *only* for diagnostics is
                                          >silly.

                                          When you put it like that how can I argue?

                                          > > I want tests to be repeatable and diagnostic tools to be very
                                          > > flexible. Finding and removing a bug is a one-shot event. It need not be
                                          > > repeatable.
                                          >
                                          >If you don't have a repeatable, automated test to prove that it never
                                          >reappears then something is missing. I.e., you might use the debugger to
                                          >step through some code to spelunk the bug while I might write some
                                          >diagnostic code to do the same. Either way, there must be a test written
                                          >which clearly drives that bug.

                                          I agree entirely. I just think that I can always do that using public members.

                                          >My guess is that our only fundamental disagreement on this point is what we
                                          >each will mean by "clearly drives that bug". :-) You might be comfortable
                                          >with just relying on the fact that an existing unit test failed and if it
                                          >fails again then you (or your team members :-) will just re-spelunk the
                                          >bug.

                                          That is correct.

                                          >I, on the other hand, want a bit more precision. So, within reason,
                                          >it sounds like it's really just come down to a matter of style.

                                          That may be. Although I think that the principals behind encapsulation are
                                          at work here. Any time you write code that breaks the encapsulation of a
                                          class you create a dependency on the implementation of the class which in
                                          turn makes the class harder to change.

                                          > >> IME, if an issue comes up where it's really worth the time/effort in
                                          > >> cranking up the debugger and stepping through the code then it's
                                          > >> critical to update/add new unit tests.
                                          >
                                          > > My experience is that if a bug is that hard to find it's generally
                                          > > because I've written too much code that isn't covered by a test. Then
                                          > > it's time to step back and find a way to make things go in smaller, more
                                          > > numerous steps, so I can test each one. I often find myself introducing
                                          > > an auxiliary class at this time. I do not add tests for my private
                                          > > members, although I do, of course, add tests for my new classes which
                                          > > often contain functionality that used to be private. So I think that our
                                          > > experiences are similar although we have chosen to respond to them
                                          > > differently.
                                          >
                                          >A point similar to one that I made in the parent thread of this thread. :-)

                                          I missed it.

                                          >I'll point out that I've made no argument about the level of difficulty of
                                          >spelunking a bug. While I certainly agree that that's a factor, my
                                          >perspective is that whether or not to test a given private method is often
                                          >a more subtle matter of the "grip" available to drive the testing via the
                                          >public interface. I.e., if the public interface gives you all of the grip
                                          >you need to drive everything then rock on! If it doesn't then some
                                          >white-box tests may be appropriate, refactoring may be appropriate, and
                                          >even ignoring it may be appropriate.

                                          I think I'm with you. Except of course that I'm advocating the refactoring
                                          over the white-box (i.e. private member) testing.

                                          > I'm not at all arguing that you should
                                          >necessarily make all of an implementation "perfect". I'm arguing against
                                          >your use of the "stability" judgement to refrain from adding regression
                                          >*tests* which drive discovered flaws.

                                          I agree. I just think that those test should go through some public interface.

                                          >W.r.t. the notion that testing private methods violates encapsulation.
                                          >I'll go so far as to say that depends on how you're testing them. I'm an
                                          >advocate of both white- and black-box testing. For example, in a Java
                                          >context, I write tests specifically against the defined Java interface
                                          >(i.e., black box) to cover the "contract" of the interface and then I write
                                          >additional tests, as needed, for each class implementing that interface
                                          >(which usually ends up as a mix of white- and black-box tests) which cover
                                          >whatever makes that class unique from the pure contract.

                                          That sounds great to me. But if you start using reflection or JNI to start
                                          digging around in the privates of those classes I think that you're asking
                                          for trouble -- unless of course that's what the classes were designed for.

                                          >[If JavaSoft had
                                          >done this sort of thing (including WTTF, etc.), just think how much better
                                          >all of Java and the Java libraries would actually be. Sigh.]

                                          Sigh indeed. I work here.

                                          >W.r.t. changing tests due to shifting behavior of the implementation. All
                                          >I can think of is that you're making an incorrect presumption. I *never*
                                          >change (nor advocate changing) the accessibility of a method merely for
                                          >testing purposes. That's Just Plain Wrong(tm). It's quite easy in e.g.,
                                          >Java to write tests for private methods without violating encapsulation or
                                          >modifying the public interface of the class.

                                          Through reflection?

                                          >Also, tests aren't ordained
                                          >to be static and unchanging -- if the implementation changes then you
                                          >should change the implementation specific tests appropriately; if the
                                          >black-box tests break then either your changing the contract (and thus the
                                          >appropriate black-box tests need to be modified/added) or your new
                                          >implementation needs work.

                                          Tests on public methods will necessarily change less often than tests on
                                          private methods. Those changes will also be more meaningful since similar
                                          changes will have to be made in all the clients of the changed member.
                                          ------------------------------------------------------------------------------------------------------------
                                          Phil Goodwin, Java Software, Sun Microsystems, 408-517-6951, or x66951

                                          "The significant problems we face cannot be solved at the same level of
                                          thinking we were at when we created them." -- Albert Einstein
                                        • Michael D. Hill
                                          Joi... To include your info as a data point in my own researches on the issue, let me ask you a few questions: Was there any point of time when the code of the
                                          Message 20 of 20 , Oct 1, 2000
                                          • 0 Attachment
                                            Joi...


                                            To include your info as a data point in my own researches on the issue, let
                                            me ask you a few questions:

                                            Was there any point of time when the code of the method was under a unit
                                            test, i.e. before it was extracted into a private method?

                                            At what point did the method become private?

                                            Why did you choose to make it private?

                                            >Don't tell me testing non-public methods has no value. If I had
                                            >followed my usual practice of testing new methods, I wouldn't have
                                            >wasted two days this week over such a simple bug.

                                            I'm not sold, BUT, I can at least offer you this: The purpose of unit
                                            testing is to maximize the developer's mobility. I'm okay with whatever
                                            rules of thumb you use to decide which tests will meet that goal.

                                            Seeya!
                                            Hill





                                            >--
                                            >Joi Ellis Software Engineer
                                            >Aravox Technologies joi@..., gyles19@...
                                            >
                                            >No matter what we think of Linux versus FreeBSD, etc., the one thing I
                                            >really like about Linux is that it has Microsoft worried. Anything
                                            >that kicks a monopoly in the pants has got to be good for something.
                                            > - Chris Johnson
                                            >
                                            >To Post a message, send it to: extremeprogramming@...
                                            >
                                            >To Unsubscribe, send a blank message to:
                                            >extremeprogramming-unsubscribe@...
                                            >
                                            >Ad-free courtesy of objectmentor.com

                                            +----------------------------------------------------------+
                                            |Michael Hill -- www.objectmentor.com
                                            |Software-> Developer, Consultant, Teacher, Coach
                                            |Lifeware-> Egghead, Romantic, Grandpa, Communitarian
                                            |<uly_REMOVE_THIS_PART_@...>
                                            +----------------------------------------------------------+
                                          Your message has been successfully submitted and would be delivered to recipients shortly.