Loading ...
Sorry, an error occurred while loading the content.

Re: VS: [XP] Test Driven Exceptions

Expand Messages
  • Jeff Grigg
    ... And NUnit, for .NET directly supports tests that expect exceptions, making it even easier.
    Message 1 of 15 , Jun 1, 2004
    • 0 Attachment
      >>> --- Phlip wrote:
      >>>> And who test-firsts every one of their exceptions?

      >> --- Adrian Howard <adrianh@q...> wrote:
      >>> I do. Does this make me weird?
      >>
      >> I'm weird too! ;->
      >> - jeff

      --- lasse.koskela@a... wrote:
      > Are you guys saying that you never write a catch clause
      > unless you've got a test that also causes it to go off?
      > What if you need to use a method that throws a checked
      > exception, say, Integer.parseInt(String)? I don't see
      > a way past the compiler...

      Is it all that hard? I do this so often, it's practically a pattern:

      | import junit.framework.TestCase;
      | public class TestParseInt extends TestCase {
      | public void testParseInt() {
      | assertEquals(0, parseInt("0"));
      | assertEquals(3, parseInt("3"));
      | assertEquals(-5, parseInt("-5"));
      | }
      | public void testBadInt() {
      | boolean caughtNumberFormatException = false;
      | try {
      | parseInt("This is not an integer.");
      | } catch (final NumberFormatException ex) {
      | caughtNumberFormatException = true;
      | }
      | assertTrue(caughtNumberFormatException);
      | }
      | private static int parseInt(final String value) {
      | return Integer.parseInt(value);
      | }
      | }

      And NUnit, for .NET directly supports tests that expect exceptions,
      making it even easier.
    • Chip Whitmer
      ... From: Jeff Grigg To: Sent: Tuesday, June 01, 2004 9:05 AM Subject: Re: VS: [XP] Test Driven
      Message 2 of 15 , Jun 1, 2004
      • 0 Attachment
        ----- Original Message -----
        From: "Jeff Grigg" <jeffgrigg@...>
        To: <extremeprogramming@yahoogroups.com>
        Sent: Tuesday, June 01, 2004 9:05 AM
        Subject: Re: VS: [XP] Test Driven Exceptions


        > --- lasse.koskela@a... wrote:
        > > Are you guys saying that you never write a catch clause
        > > unless you've got a test that also causes it to go off?
        > > What if you need to use a method that throws a checked
        > > exception, say, Integer.parseInt(String)? I don't see
        > > a way past the compiler...
        >
        > Is it all that hard? I do this so often, it's practically a pattern:
        >
        > | import junit.framework.TestCase;
        > | public class TestParseInt extends TestCase {
        > | public void testParseInt() {
        > | assertEquals(0, parseInt("0"));
        > | assertEquals(3, parseInt("3"));
        > | assertEquals(-5, parseInt("-5"));
        > | }
        > | public void testBadInt() {
        > | boolean caughtNumberFormatException = false;
        > | try {
        > | parseInt("This is not an integer.");
        > | } catch (final NumberFormatException ex) {
        > | caughtNumberFormatException = true;
        > | }
        > | assertTrue(caughtNumberFormatException);
        > | }
        > | private static int parseInt(final String value) {
        > | return Integer.parseInt(value);
        > | }
        > | }

        Or the simpler:

        public void testBadInt() {
        try {
        parseInt( "This is not an integer" );
        fail( "Should have thrown NumberFormatException" );
        } catch( NumberFormatException correctException )
        {}
        }

        When the exception being tested is generated by my own code, I typically do
        a more detailed check on the exception itself:

        public void testLoadIndex() {
        File badFile = new File( "notFound.xml" );
        try {
        loadIndex( badFile );
        fail( "Should have thrown FileNotFoundException" );
        } catch( FileNotFoundException correctException ) {
        assertEquals( badFile.toString(), correctException.getMessage() );
        }
        }

        These tests *must* be done test-first. It's easy to forget to add the
        "fail()" line, in which case existing code will pass the test even if no
        exception is thrown. So you have to see the test fail before you add the
        code to throw the exception.

        Now that I think about it: if you're not doing it test-first, then I guess
        the boolean mechanism above is safer than my "simpler" example.

        - Chip Whitmer
      • Jeff Grigg
        ... Maybe it s a bad habit, but I sometimes test assertion error exceptions into the code, so a fail() method in the try block will, in some cases, get
        Message 3 of 15 , Jun 1, 2004
        • 0 Attachment
          --- "Chip Whitmer" <cwhitmer@m...> wrote:
          > Now that I think about it: if you're not doing it test-
          > first, then I guess the boolean mechanism above is safer
          > than my "simpler" example.

          Maybe it's a bad habit, but I sometimes test assertion error
          exceptions into the code, so a 'fail()' method in the 'try' block
          will, in some cases, get caught and ignored by my own 'catch' blocks.
        • Chip Whitmer
          ... From: Jeff Grigg To: Sent: Tuesday, June 01, 2004 1:12 PM Subject: Re: VS: [XP] Test Driven
          Message 4 of 15 , Jun 1, 2004
          • 0 Attachment
            ----- Original Message -----
            From: "Jeff Grigg" <jeffgrigg@...>
            To: <extremeprogramming@yahoogroups.com>
            Sent: Tuesday, June 01, 2004 1:12 PM
            Subject: Re: VS: [XP] Test Driven Exceptions


            > --- "Chip Whitmer" <cwhitmer@m...> wrote:
            > > Now that I think about it: if you're not doing it test-
            > > first, then I guess the boolean mechanism above is safer
            > > than my "simpler" example.
            >
            > Maybe it's a bad habit, but I sometimes test assertion error
            > exceptions into the code, so a 'fail()' method in the 'try' block
            > will, in some cases, get caught and ignored by my own 'catch' blocks.

            Yes -- testing the assertions themselves is the other "gotcha," and is the
            other reason why I added the checks on the caught exception:

            public void testAssertDouble() {
            try {
            assertDouble( 0.3, (double) 2 / 5 );
            fail( "assertDouble() should have failed" );
            } catch( AssertionFailedError x ) {
            assertEquals( "expected:<0.3> but was:<0.4>", x.getMessage() );
            }
            }

            I still find this more expressive than the boolean approach, but it's
            probably just personal preference at this point.

            I will grant that the failure that is finally reported by JUnit is a little
            confusing -- something like this:
            "expected:<expected:<0.3> but was:<0.4>> but was:<assertDouble() should
            have failed>"

            - Chip Whitmer
          • lasse.koskela@accenture.com
            ... No and umm... kind of . This particular conversation started from someone stating that he never writes a single line of code without a test for it (or
            Message 5 of 15 , Jun 2, 2004
            • 0 Attachment
              > Lasse:
              > > ....you would've actually executed the catch-block, but who writes the
              > > exceptional cases first? I don't. Should I? I don't think so.
              > John Mitchell:
              > Please elaborate. Do you only care about the system working when things
              > are perfect?
              >
              > Or are you saying that you consciously violate test-first and that's okay

              > because it's (only) for exceptional cases? In that case, do you actually
              > write the exceptional tests immediately after you wrote the code?



              "No" and "umm... kind of".

              This particular conversation started from someone stating that he "never" writes a single line of code without a test for it (or something along those lines) and I went into nitpicking mode (a decent way of learning if the other side of the conversation plays along ;-)...

              My point is that Java, for example, forces you to write some exception handling code without a test (i.e. statement coverage < 100%) unless you *start* with the exceptional case instead of the normal case. Personally, I write the normal case first because that's what I'm most interested about -- it's more valuable than a piece of code that handles the problems well but doesn't fulfill the actual function -- which inevitably leads to having some catch-clauses without matching tests (*until I get to writing the tests for those exceptional cases*). Obviously the time frame isn't too long, but it's still not technically test-first.

              I'm okay with it as long as I don't know of a better way.

              - Lasse -



              This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.


              [Non-text portions of this message have been removed]
            • lasse.koskela@accenture.com
              ... Ah, but your tests still aren t branching the *execution* into that catch block ;) ... Touche... - Lasse - This message is for the designated recipient
              Message 6 of 15 , Jun 3, 2004
              • 0 Attachment
                Robert Watkins:
                > you do it like this:
                > | public int toInt(String s) {
                > | try {
                > | return Integer.parseInt(s);
                > | } catch (NumberFormatException e) {
                > | throw new IllegalArgumentException(s + " is not valid");
                > | }
                > | }

                Ah, but your tests still aren't branching the *execution* into that catch block ;)

                Robert Watkins:
                > [having the method initially throw a RuntimeException] means I can get a red bar on...

                Touche...

                - Lasse -



                This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.


                [Non-text portions of this message have been removed]
              • lasse.koskela@accenture.com
                ... Good point. It certainly is a design decision and does require a test to make it explicit. - Lasse - This message is for the designated recipient only and
                Message 7 of 15 , Jun 3, 2004
                • 0 Attachment
                  Robert Watkins:
                  > Mind you, you should pay attention to a statement you made earlier: the
                  > method "has a contract that it shouldn't throw any (checked) exceptions".
                  > That's a design choice you made, and without a test. I would argue that the
                  > default decision should be to propagate the exception, which is the minimal
                  > amount you need to do to get compilation. Deciding to handle the exception
                  > internally should require a test.

                  Good point. It certainly is a design decision and does require a test to make it explicit.

                  - Lasse -


                  This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.


                  [Non-text portions of this message have been removed]
                • lasse.koskela@accenture.com
                  ... Yep. This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it
                  Message 8 of 15 , Jun 3, 2004
                  • 0 Attachment
                    Ron:
                    > Well, this is why I hate those exception thingies so much. But one could
                    > have written the original version with "throws", and then write a test
                    > removing the throw?

                    Yep.


                    This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.


                    [Non-text portions of this message have been removed]
                  • John D. Mitchell
                    ... [...] ... For the sake of argument, I ll take that last statement to mean that you write the tests for the newly created exceptional cases immediately
                    Message 9 of 15 , Jun 3, 2004
                    • 0 Attachment
                      >>>>> "lasse" == lasse koskela <lasse.koskela@...> writes:
                      [...]

                      >>> ....you would've actually executed the catch-block, but who writes the
                      >>> exceptional cases first? I don't. Should I? I don't think so.

                      >> Please elaborate. Do you only care about the system working when things
                      >> are perfect?

                      >> Or are you saying that you consciously violate test-first and that's
                      >> okay because it's (only) for exceptional cases? In that case, do you
                      >> actually write the exceptional tests immediately after you wrote the
                      >> code?

                      > "No" and "umm... kind of".

                      > This particular conversation started from someone stating that he "never"
                      > writes a single line of code without a test for it (or something along
                      > those lines) and I went into nitpicking mode (a decent way of learning if
                      > the other side of the conversation plays along ;-)...

                      :-)


                      > My point is that Java, for example, forces you to write some exception
                      > handling code without a test (i.e. statement coverage < 100%) unless you
                      > *start* with the exceptional case instead of the normal case. Personally,
                      > I write the normal case first because that's what I'm most interested
                      > about -- it's more valuable than a piece of code that handles the
                      > problems well but doesn't fulfill the actual function -- which inevitably
                      > leads to having some catch-clauses without matching tests (*until I get
                      > to writing the tests for those exceptional cases*). Obviously the time
                      > frame isn't too long, but it's still not technically test-first.

                      For the sake of argument, I'll take that last statement to mean that you
                      write the tests for the newly created exceptional cases immediately after
                      making the "normal" case work.

                      > I'm okay with it as long as I don't know of a better way.

                      Part of the larger question is where the line is drawn between the
                      pro-activeness of "test first design" driven test writing and, in your
                      example, the reactive "high-quality software" driven test writing
                      (motivated by the concern for high-percentage code coverage).

                      Obviously (I hope), it's possible to take a completely pro-active stance.

                      However, in terms of the more blended approaches, one technique that hasn't
                      been mentioned so far is the "backtracking is okay, listen to what the code
                      is telling you approach". Basically, in your example, once you started
                      writing that code which would introduce the exceptional cases, you could
                      backtrack (out of the existing "normal" code test (put it on the top of
                      your task stack)) and write the necessary test(s) just for the exceptional
                      case(s) and then come back to the "normal" case test and continue.

                      Have fun,
                      John
                    • lasse.koskela@accenture.com
                      ... Yes. Immediately being in the range of a couple of minutes. ... I hadn t thought of that. My initial gut feeling is that such backtracking would still be
                      Message 10 of 15 , Jun 3, 2004
                      • 0 Attachment
                        > > Lasse:
                        > > ... which inevitably
                        > > leads to having some catch-clauses without matching tests (*until I get
                        > > to writing the tests for those exceptional cases*). Obviously the time
                        > > frame isn't too long, but it's still not technically test-first.
                        > John:
                        > For the sake of argument, I'll take that last statement to mean that you
                        > write the tests for the newly created exceptional cases immediately after
                        > making the "normal" case work.

                        Yes. "Immediately" being in the range of a couple of minutes.

                        > John:
                        > Part of the larger question is where the line is drawn between the
                        > pro-activeness of "test first design" driven test writing and, in your
                        > example, the reactive "high-quality software" driven test writing
                        > (motivated by the concern for high-percentage code coverage).
                        >
                        > Obviously (I hope), it's possible to take a completely pro-active stance.
                        >
                        > However, in terms of the more blended approaches, one technique that hasn't
                        > been mentioned so far is the "backtracking is okay, listen to what the code
                        > is telling you approach". Basically, in your example, once you started
                        > writing that code which would introduce the exceptional cases, you could
                        > backtrack (out of the existing "normal" code test (put it on the top of
                        > your task stack)) and write the necessary test(s) just for the exceptional
                        > case(s) and then come back to the "normal" case test and continue.

                        I hadn't thought of that. My initial gut feeling is that such backtracking would still be a bit backwards in terms of importance -- the way I see it, the normal case has more value to the customer than the exceptional cases.

                        - Lasse -



                        This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.


                        [Non-text portions of this message have been removed]
                      • John D. Mitchell
                        ... [...] ... IME, it depends completely on the focus of the client. However, IMO, it should depend (much more) on the nature of the criticality of the
                        Message 11 of 15 , Jun 3, 2004
                        • 0 Attachment
                          >>>>> "lasse" == lasse koskela <lasse.koskela@...> writes:
                          [...]

                          >> However, in terms of the more blended approaches, one technique that
                          >> hasn't been mentioned so far is the "backtracking is okay, listen to
                          >> what the code is telling you approach". Basically, in your example, once
                          >> you started writing that code which would introduce the exceptional
                          >> cases, you could backtrack (out of the existing "normal" code test (put
                          >> it on the top of your task stack)) and write the necessary test(s) just
                          >> for the exceptional case(s) and then come back to the "normal" case test
                          >> and continue.

                          > I hadn't thought of that. My initial gut feeling is that such
                          > backtracking would still be a bit backwards in terms of importance -- the
                          > way I see it, the normal case has more value to the customer than the
                          > exceptional cases.

                          IME, it depends completely on the focus of the client. However, IMO, it
                          should depend (much more) on the nature of the criticality of the software.
                          I.e., the primary focus of life critical software should be "first, don't
                          kill anybody".

                          Take care,
                          John
                        • lasse.koskela@accenture.com
                          ... ...except that this won t compile ;) ... Wow. So it seems. I picked a bad example after all. I m so used to expect a NumberFormatException that I ve
                          Message 12 of 15 , Jun 3, 2004
                          • 0 Attachment
                            JB:
                            > ...or...
                            >
                            > | public int toInt(String s) {
                            > | try {
                            > | return Integer.parseInt(s);
                            > | } catch (NumberFormatException ignored) {
                            > | }
                            > | }

                            ...except that this won't compile ;)

                            JB:
                            > (By the way... NumberFormatException is unchecked, so I don't even have
                            > to catch it!)

                            Wow. So it seems. I picked a bad example after all.
                            I'm so used to "expect" a NumberFormatException that I've subconsciously made it a "checked" exception :)

                            - Lasse -



                            This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.


                            [Non-text portions of this message have been removed]
                          • J. B. Rainsberger
                            ... Robert Watkins pointed that out, too. As a wrote to him, since I must return something, and int doesn t allow me to return a value representing
                            Message 13 of 15 , Jun 4, 2004
                            • 0 Attachment
                              lasse.koskela@... wrote:
                              > JB:
                              >
                              >>...or...
                              >>
                              >>| public int toInt(String s) {
                              >>| try {
                              >>| return Integer.parseInt(s);
                              >>| } catch (NumberFormatException ignored) {
                              >>| }
                              >>| }
                              >
                              > ...except that this won't compile ;)

                              Robert Watkins pointed that out, too. As a wrote to him, since I must
                              return something, and "int" doesn't allow me to return a value
                              representing "nothing," the method signature is inadequate to the
                              current task. Either I have to change it or add extra stuff to mollify
                              the compiler.

                              >
                              > JB:
                              >>(By the way... NumberFormatException is unchecked, so I don't even have
                              >>to catch it!)
                              >
                              > Wow. So it seems. I picked a bad example after all.
                              > I'm so used to "expect" a NumberFormatException that I've subconsciously made it a "checked" exception :)

                              Someone else suggested catching NFE and throwing
                              IllegalArgumentException, so I checked it out: NFE /is/ an IAE. :)
                              (Makes sense from the names, no?)
                              --
                              J. B. Rainsberger,
                              Diaspar Software Services
                              http://www.diasparsoftware.com :: +1 416 791-8603
                              Let's write software that people understand
                            Your message has been successfully submitted and would be delivered to recipients shortly.