Loading ...
Sorry, an error occurred while loading the content.

[XP] Re: Writing Unit test to existing code

Expand Messages
  • ericheikkila@yahoo.com
    ... replace ... more ... case unit ... to without ... AGREED! :) -Eric
    Message 1 of 14 , May 2, 2001
    • 0 Attachment
      --- In extremeprogramming@y..., "Jason Rogers" <jason.rogers@t...>
      wrote:
      >
      > > That sounds like good advice. As I read this, it occurred
      > > to me that if you
      > > want to enhance an already working program that has no
      > > tests, the best route
      > > to take is probably not to try and retrofit unit tests,
      > > but just write
      > > acceptance tests. After all, your overriding fear is that
      > > the customer will
      > > get a new program that has lost functionality, and to
      > > address that fear, all
      > > you really need is acceptance tests.
      >
      > Acceptance tests are fine, and needful. But I don't think they can
      replace
      > the sturdiness of unit tests. It seems to me that there should be
      more
      > concern for the danger of existing bugs being persisted, in which
      case unit
      > tests ought to be "retro-fitted." Can you have high percentage of
      > confidence that a system is really working the way it is supposed
      to without
      > that system having unit tests? I don't know if I can.

      AGREED! :)

      -Eric
    • Brian C. Robinson
      ... This looks like a case where you would build a test that would demonstrate the bug, then fix the bug with the test to prove you ve fixed it.
      Message 2 of 14 , May 2, 2001
      • 0 Attachment
        At 08:52 AM 5/2/01, you wrote:
        >(If they have no workaround however then it's a showstopper. Showstopper
        >bugs need to be fixed immediately and developers and users are "willing" to
        >tolerate the risk of introducing new bugs to get those fixed.)

        This looks like a case where you would build a test that would demonstrate
        the bug, then fix the bug with the test to prove you've fixed it.
      • Arrizza, John
        ... SOP.
        Message 3 of 14 , May 2, 2001
        • 0 Attachment
          > -----Original Message-----
          > From: Brian C. Robinson [mailto:brian.c.robinson@...]
          > This looks like a case where you would build a test that
          > would demonstrate
          > the bug, then fix the bug with the test to prove you've fixed it.

          SOP.
        • wecaputo@thoughtworks.com
          ... I for one, agree with this. If you don t have Unit Tests, Acceptance Tests can really help tell things are broken -- but it can be really hard to find
          Message 4 of 14 , May 2, 2001
          • 0 Attachment
            Peter van Rooijen:
            >If you then break an acceptance test, finding the mistake will be harder
            >without unit tests, but especially if you can run the acceptance tests
            >frequently, it sounds like a smart trade-off.

            I for one, agree with this.

            If you don't have Unit Tests, Acceptance Tests can really help tell things
            are broken -- but it can be really hard to find what broke. At one of our
            client's, there is a good bit of code nicely covered by Unit Tests, some
            good quality code with no unit tests, and some very poor quality code with
            next to no unit tests.

            Once the Acceptance tests were up and running (and failing), What we found
            was:

            a)The unit tested code was responsible for the fewest defect resolution
            tasks. When there were some, they were generally easy to find and fix.

            b)The good quality code that had no unit tests had quite a few defects, but
            they were generally easy to fix when found, but sometimes very difficult to
            isolate.

            c)The poor quality code with next to no unit tests had the most defects.
            These were the hardest to isolate, and some are still not fixed. The unit
            tests for this code were red herrings, they gave no accurate feedback on
            the state of the code, but provided warm fuzzies for the original authors
            of the code, and their manager. Unfortunately, they give a false sense of
            security to people modifying that code.

            Finding the reason an AT was failing meant debug traces -- lots of them,
            through many layers of code. It was time consuming and error prone. Worse,
            it often took bug-hunters several hours to get back to where they left off
            the day before because defects were being introduced and not caught (still
            not enough tests, and some of the client's developers still don't like to
            write Unit Tests).

            Two good things came of this, the system slowly got more Unit Tests, and
            those who did the tracing really know the work flow of the system ;-)

            Best,
            Bill
          • azami@speakeasy.net
            ... defects. ... unit ... feedback on ... authors ... sense of ... Okay, that s an incendiary subject line. It should be bad unit tests considered harmful .
            Message 5 of 14 , May 2, 2001
            • 0 Attachment
              --- In extremeprogramming@y..., wecaputo@t... wrote:
              > c)The poor quality code with next to no unit tests had the most
              defects.
              > These were the hardest to isolate, and some are still not fixed. The
              unit
              > tests for this code were red herrings, they gave no accurate
              feedback on
              > the state of the code, but provided warm fuzzies for the original
              authors
              > of the code, and their manager. Unfortunately, they give a false
              sense of
              > security to people modifying that code.

              Okay, that's an incendiary subject line. It should be "bad unit tests
              considered harmful". But this _is_ a counter-point to the oft-cited
              "axiom" that any unit tests are better than none.

              How do we identify bad unit tests? Is there some way we can know
              whether a set of unit tests should NOT be giving us confidence?

              As I am new to writing unit tests, and especially since I don't have a
              partner, I am stricken by the fear that I'm writing bad tests and
              getting a false sense of security.

              Fear is bad, so how can I assure myself? Do I just have to see what
              bugs crop up, and discover what I should have tested that way? Is UT
              experience the only answer?

              -Matthew
              azami@...
            • Robert Watkins
              ... The warm fuzzies is a real danger in PrettyAdventurousProgramming. It s like walking a tightrope with a safety net that s only rated for 10 kilos; it
              Message 6 of 14 , May 2, 2001
              • 0 Attachment
                Bill Caputo wrote:
                > c)The poor quality code with next to no unit tests had the
                > most defects.
                > These were the hardest to isolate, and some are still not
                > fixed. The unit
                > tests for this code were red herrings, they gave no accurate
                > feedback on
                > the state of the code, but provided warm fuzzies for the
                > original authors
                > of the code, and their manager. Unfortunately, they give a
                > false sense of
                > security to people modifying that code.

                The "warm fuzzies" is a real danger in PrettyAdventurousProgramming. It's
                like walking a tightrope with a safety net that's only rated for 10 kilos;
                it looks nice and safe, but when you fall, the net doesn't help you.

                One of the things I try to get people to learn with unit tests is to be
                aware of when you're making a change that _should_ break tests. If making
                such a change doesn't break tests, or require tests to be changed, alarm
                bells should ring; it means either that your tests aren't extensive enough,
                or that you don't understand your system as well as you think.

                One of these days, I'm going to look into something like Jester...

                Robert.

                --
                "Duct tape is like the Force: it has a light side, a dark side,
                and it holds the universe together"
                Robert Watkins robertdw@... robertw@...


                [Non-text portions of this message have been removed]
              • jbrewer@jera.com
                ... a ... UT ... Deliberately break your code. Verify that the appropriate tests fail. Fix your code (or just revert to the previous version). Verify that
                Message 7 of 14 , May 2, 2001
                • 0 Attachment
                  --- In extremeprogramming@y..., azami@s... wrote:
                  > How do we identify bad unit tests? Is there some way we can know
                  > whether a set of unit tests should NOT be giving us confidence?
                  >
                  > As I am new to writing unit tests, and especially since I don't have
                  a
                  > partner, I am stricken by the fear that I'm writing bad tests and
                  > getting a false sense of security.
                  >
                  > Fear is bad, so how can I assure myself? Do I just have to see what
                  > bugs crop up, and discover what I should have tested that way? Is
                  UT
                  > experience the only answer?

                  Deliberately break your code. Verify that the appropriate tests fail.
                  Fix your code (or just revert to the previous version). Verify that
                  the test all pass. I do this whenever I feel a crisis in confidence
                  coming on.

                  There is actually a tool called "Jester" that introduces faults into
                  your code, and reports if the associated tests failed or not.

                  http://www.xpdeveloper.com/cgi-bin/wiki.cgi?JesTer

                  John Brewer
                  Jera Design

                  XP FAQ: http://www.jera.com/techinfo/xpfaq.html
                • Arrizza, John
                  ... broke. Agreed. But it s difficult to find what s broke without ATs as well, i.e. I m no worse off by adding ATs. A large benefit of doing ATs before adding
                  Message 8 of 14 , May 8, 2001
                  • 0 Attachment
                    > -----Original Message-----
                    > From: wecaputo@... [mailto:wecaputo@...]
                    > If you don't have Unit Tests, Acceptance Tests can really
                    > help tell things are broken -- but it can be really hard to find what
                    broke.
                    Agreed. But it's difficult to find what's broke without ATs as well, i.e.
                    I'm no worse off by adding ATs.

                    A large benefit of doing ATs before adding UTs is that they allow me to
                    refactor (in a limited, somewhat-safe way), which allows me to add UTs at a
                    much lower cost, which is where I really want to be.

                    Also, ATs add test coverage where there was none before. That's A Good
                    Thing. If I eventually add UTs across the board or just in a few select
                    places, the ATs will help in that effort.

                    John
                  • azami@speakeasy.net
                    ... what ... i.e. ... Also, from a refactoring perspective, you start with passing ATs, and re-run after each small change. It s not difficult to find what
                    Message 9 of 14 , May 8, 2001
                    • 0 Attachment
                      --- In extremeprogramming@y..., "Arrizza, John" <john.arrizza@m...>
                      wrote:
                      > > -----Original Message-----
                      > > From: wecaputo@t... [mailto:wecaputo@t...]
                      > > If you don't have Unit Tests, Acceptance Tests can really
                      > > help tell things are broken -- but it can be really hard to find
                      what
                      > broke.
                      > Agreed. But it's difficult to find what's broke without ATs as well,
                      i.e.
                      > I'm no worse off by adding ATs.

                      Also, from a refactoring perspective, you start with passing ATs, and
                      re-run after each small change. It's not difficult to find what broke
                      - it's whatever you just did. Not as good as UTs, but as Mr. Arrizza
                      points out, it gives us a way to reduce the cost of adding UTs.

                      -Matthew
                      azami@...
                    Your message has been successfully submitted and would be delivered to recipients shortly.