Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] A TDD Failure

Expand Messages
  • Andrew Wall
    Kent, I can t possibly demonstrate undefined behaviour, but what I can do is keep the code within the bounds of defined behaviour. I can write a test that
    Message 1 of 5 , Mar 1, 2005
    • 0 Attachment
      Kent,
      I can't possibly demonstrate undefined behaviour, but what I can do
      is keep the code within the bounds of defined behaviour.
      I can write a test that attempts to overfill the string in question
      and test for the string ending up being filled exactly to the end.
      Then I can remove any current behaviour I've put in (Yes, that bit
      was not written Test First) and see the test fail. (I'm fairly sure
      that it won't bring down the test project, but I can never be
      sure.) I can then put some behaviour back in to demonstrate the
      test passing.

      I'll do that tomorrow and report on it later.

      Andrew Wall
      Still not doing XP

      --- In extremeprogramming@yahoogroups.com, "Kent Beck" <kentb@e...>
      wrote:
      > Andrew,
      >
      > Thank you for following up on this. I would like to see a test that
      > demonstrates the "undefined" behavior you (accidentally) expected
      and
      > another that demonstrates the behavior you got in the production
      > environment.
      >
      > Kent Beck
      > Three Rivers Institute
      >
      > > -----Original Message-----
      > > From: Andrew Wall [mailto:ajwall.geo@y...]
      > > Sent: Saturday, February 26, 2005 11:35 AM
      > > To: extremeprogramming@yahoogroups.com
      > > Subject: Re: [XP] A TDD Failure
      > >
      > > Follow-up
      > >
      > > Thanks for the replies.
      > > I've not been able to reproduce the crash of the target machine
      in
      > > the test project, although I've been able to avoid the crash on
      the
      > > target machine now. It was something to do with an unchecked
      string
      > > length. So 'undefined behaviour' is different between the two
      > > compilers.
      > >
      > > Anyway, that was just a little bump on the road of practising
      TDD
      > > and not a hiatus.
      > >
      > > Andrew Wall
      > > Still not doing XP
    • Andrew Wall
      Next report: The first thing I did was to remove the patch that avoided the crash on the target machine. However, no tests failed! Oops! Well I knew that
      Message 2 of 5 , Mar 4, 2005
      • 0 Attachment
        Next report:

        The first thing I did was to remove the patch that avoided the crash
        on the target machine. However, no tests failed! Oops! Well I
        knew that was going to happen didn't I.
        I then wrote a new test that fed a long string through the checking
        process. The idea being that this would be stored, in fact with the
        patch removed, this would overrun the fixed length buffer. The test
        then asked for the stored string and asserted that its length was at
        maximum. Ah ha! Now here was a test that failed.

        Since my last post I was thinking about what should happen, and I
        realised that the concept of the fixed length buffer should be
        encapsulated, so now I knew a better patch to apply (i.e. use strnlen
        () ). So in it went and the test passed.
        I recompiled for the target machine and that worked also. Job done!
        Now I am even more confident of TDD: I've encountered a problem and
        solved it using TDD, even though at first I thought TDD had let me
        down.

        Some of my colleagues are a bit paranoid about software: They think
        that even recompiling may vary the behaviour of software. They may
        be right if you change some libraries that you compile with, but TDD
        should expose that. I'm more confident in the libraries that come
        with the two different compilers that I have. If I had a reason to,
        I could probably compile the tests with the target compiler and run
        those under a console app version of my test framework, possibly as
        part of the build process. But 3000+ tests tell me that the time is
        not now.

        Andrew Wall
        Still not doing XP


        --- In extremeprogramming@yahoogroups.com, "Andrew Wall"
        <ajwall.geo@y...> wrote:
        >
        > Kent,
        > I can't possibly demonstrate undefined behaviour, but what I can
        do
        > is keep the code within the bounds of defined behaviour.
        > I can write a test that attempts to overfill the string in
        question
        > and test for the string ending up being filled exactly to the
        end.
        > Then I can remove any current behaviour I've put in (Yes, that bit
        > was not written Test First) and see the test fail. (I'm fairly
        sure
        > that it won't bring down the test project, but I can never be
        > sure.) I can then put some behaviour back in to demonstrate the
        > test passing.
        >
        > I'll do that tomorrow and report on it later.
        >
        > Andrew Wall
        > Still not doing XP
        >
        > --- In extremeprogramming@yahoogroups.com, "Kent Beck"
        <kentb@e...>
        > wrote:
        > > Andrew,
        > >
        > > Thank you for following up on this. I would like to see a test
        that
        > > demonstrates the "undefined" behavior you (accidentally)
        expected
        > and
        > > another that demonstrates the behavior you got in the production
        > > environment.
        > >
        > > Kent Beck
        > > Three Rivers Institute
        > >
        > > > -----Original Message-----
        > > > From: Andrew Wall [mailto:ajwall.geo@y...]
        > > > Sent: Saturday, February 26, 2005 11:35 AM
        > > > To: extremeprogramming@yahoogroups.com
        > > > Subject: Re: [XP] A TDD Failure
        > > >
        > > > Follow-up
        > > >
        > > > Thanks for the replies.
        > > > I've not been able to reproduce the crash of the target
        machine
        > in
        > > > the test project, although I've been able to avoid the crash
        on
        > the
        > > > target machine now. It was something to do with an unchecked
        > string
        > > > length. So 'undefined behaviour' is different between the two
        > > > compilers.
        > > >
        > > > Anyway, that was just a little bump on the road of practising
        > TDD
        > > > and not a hiatus.
        > > >
        > > > Andrew Wall
        > > > Still not doing XP
      • Davide Inglima
        ... Andrew: I don t know if it would be allowed, so please disregard my post if it is impossible or unfeasible... but in my opinion you could apply the new
        Message 3 of 5 , Mar 7, 2005
        • 0 Attachment
          On Fri, 04 Mar 2005 17:05:51 -0000, Andrew Wall <ajwall.geo@...> wrote:

          > I could probably compile the tests with the target compiler and run
          > those under a console app version of my test framework, possibly as
          > part of the build process. But 3000+ tests tell me that the time is
          > not now.

          Andrew: I don't know if it would be allowed, so please disregard my
          post if it is impossible or unfeasible... but in my opinion you could
          apply the new testing framework and practices little by little to the
          new modules or to old modules that need to be debugged and modified.
          Then, when you have time, you could refactor the old tests.

          Start small, create a new testsuite and put, let's say, 10 tests under
          the automatic build. After being sure that the old tests behave
          correctly, remove those 10 tests from the old tree and convert other
          tests. Then, little by little, migrate the old tests to the new ones.

          Of course, apply this only if you dare, care and have resources to do it ;).
        • Andrew Wall
          ... run ... as ... time is ... could ... the ... modified. ... under ... other ... ones. ... do it ;). Thanks for your reply Davide. The reason to compile the
          Message 4 of 5 , Mar 11, 2005
          • 0 Attachment
            --- In extremeprogramming@yahoogroups.com, Davide Inglima
            <limacat@g...> wrote:
            > On Fri, 04 Mar 2005 17:05:51 -0000, Andrew Wall <ajwall.geo@y...>
            wrote:
            >
            > > I could probably compile the tests with the target compiler and
            run
            > > those under a console app version of my test framework, possibly
            as
            > > part of the build process. But 3000+ tests tell me that the
            time is
            > > not now.
            >
            > Andrew: I don't know if it would be allowed, so please disregard my
            > post if it is impossible or unfeasible... but in my opinion you
            could
            > apply the new testing framework and practices little by little to
            the
            > new modules or to old modules that need to be debugged and
            modified.
            > Then, when you have time, you could refactor the old tests.
            >
            > Start small, create a new testsuite and put, let's say, 10 tests
            under
            > the automatic build. After being sure that the old tests behave
            > correctly, remove those 10 tests from the old tree and convert
            other
            > tests. Then, little by little, migrate the old tests to the new
            ones.
            >
            > Of course, apply this only if you dare, care and have resources to
            do it ;).

            Thanks for your reply Davide.
            The reason to compile the tests with the target compiler is to allow
            the tests to experience the same undefined behaviour that the target
            machine might do. The time is not now because 3000+ tests tell me
            that I'm doing OK right now when I use an alternate compiler/IDE.
            If I weren't doing OK and I kept on coming up against quirky
            behaviour, I might reconsider.

            Andrew Wall
            Still not doing XP
          Your message has been successfully submitted and would be delivered to recipients shortly.