Loading ...
Sorry, an error occurred while loading the content.
 

Re: [XP] Test-first an infinite loop

Expand Messages
  • Curtis Cooley
    On Sun, Apr 4, 2010 at 2:50 PM, Bill Caputo
    Message 1 of 31 , Apr 4, 2010
      On Sun, Apr 4, 2010 at 2:50 PM, Bill Caputo <
      list-subscriber@...> wrote:

      >
      >
      > Hi All,
      >
      > This is one of those test-first situations that I'm never happy with:
      > How to implement a requirement that a process not terminate (or
      > possibly until some external stimuli like a signal or kill file is
      > triggered) test-first?
      >
      > Over the years, I've tried the following:
      >
      > 1) Write a test to assert code is running, then have test kill it in
      > some way perhaps in the way prescribed by the task at hand (probably
      > using multiple threads, yuck). And yuck for the whole approach. A)
      > doesn't show that the process will run indefinitely, only that it is
      > running when asserted to be running. B) generally slow (often needs
      > some kind of sleep or other timer to ensure process is actually
      > running before assertion).
      >
      > 2) Write a test to assert code is running for n amount of time, else
      > fail. Similar to the previous one, but this just fails if the process
      > is not running at the end of the test. Again, slow (there's some sleep
      > time built into the test) and still doesn't specify the actual
      > behavior (run until stopped), and probably using multiple threads
      > again (double yuck).
      >
      > 3) Don't test it - treat the "run forever" bit as part of the UI and
      > ensure the loop is empty of all but a call to tested code, then test
      > the hell out of the loop contents. In some ways this feels like a
      > failure, but is least expensive (both in terms of test run time, and
      > maintenance knowledge burden for the tests). The other problem with
      > this approach is that it requires either having the kill event left
      > untested as well, or faked and mocked somehow (this last part I've
      > never done, but I'd like to give it a go some day).
      >
      > In short: This type of thing smells to high heaven, but all
      > long-running processes (think services, daemons, etc) have this type
      > of behavior, so its something that comes up a lot (but not often
      > enough that I've seen a lot of discussion of it, hence my post).
      >
      > Thoughts, ideas, approaches, criticisms, all welcome.
      >
      > From a TDD perspective, you break it down into the smallest tests and
      functions you can and test them. The loop isn't tested in the unit tests
      because unit tests have to be fast. Preferably so fast that you are able to
      run the entire suite every RGR cycle and not just the test you are working
      on.

      To truly test the loop, we deployed to test servers and let them 'soak'. Not
      very agile, but for long running processes, you need to test them by letting
      them run. We'd deploy every couple of weeks and check the system every
      couple of days. Good logging of errors and failures helps this a lot. I
      didn't like this process, because we started to rely on it for things that
      should have had automated acceptance tests. Too often I'd hear "Let's throw
      it in the lab and let it 'soak'". I often wondered when we would let it
      rinse and dry ;)

      On the other hand, we often found things like memory leaks and process hogs
      by doing this. I'm not sure how we would have done this any other way. I
      suppose you could run tests on an 'accelerated' system. For example, we ran
      services every few seconds, so maybe we could have run them 10 times as
      often for a tenth as long and gotten the same results. Hard to prove,
      though.
      --
      Curtis Cooley
      curtis.cooley@...
      home:http://curtiscooley.com
      blog:http://ponderingobjectorienteddesign.blogspot.com
      ===============
      Leadership is a potent combination of strategy and character. But if you
      must be without one, be without the strategy.
      -- H. Norman Schwarzkopf


      [Non-text portions of this message have been removed]
    • Bill Michell
      Well, an infinite loop is definitely a special case construct - not something that you want to see in normal usage. Sounds like what we are looking for is a
      Message 31 of 31 , Apr 9, 2010
        Well, an infinite loop is definitely a special case construct - not something that you want to see in normal usage.

        Sounds like what we are looking for is a good use case - maybe a daemon that keeps running some processing method even when given no time by the current thread.

        So I'd make the processing method simply increment a counter.

        The first test just sets the counter to 0 and spawns a thread that runs our main method. The test then checks that the counter value eventually becomes at least 1 - and hence that our processing method got called. Note that spawning the thread is probably part of the test, not the method under test.

        No loop required - just a call to our processing method.

        The next test spawns the thread again, but this time checks the counter eventually becomes at least, say, 10.

        I'd posit that the simplest way to pass the test is to surround the worker method call with a while(1) construct. If you wanted to go via ten calls to the worker method first, knock yourself out.

        For completeness, I'd probably write a test that checks the counter with a decent time interval between the checks, and show that the counter value was continuing to increment. If you haven't reached while(1) already, then this test will surely get you there...

        Yes, it isn't a fast test, and yes, it involves spawning threads. Not pretty. But I'd argue that you are working with a special use case here - even though on the face of it, the construct is a really simple one. It also gives a nice clean design (separating the "never stop" from the "processing" stuff) that lets you do nice things like checking error conditions and termination conditions if those ever become interesting.

        On 8 Apr 2010, at 23:09, Tom wrote:

        > Eggzackly - hence my hesitation.
        >
        > --- In extremeprogramming@yahoogroups.com, Adam Sroka <adam.sroka@...> wrote:
        > >
        > > On Thu, Apr 8, 2010 at 11:06 AM, Tom <rossentj@...> wrote:
        > > >
        > > >
        > > >
        > > > I hesitate to suggest such ugly complexity - but ... maybe have the test start the target in a separate thread which it can kill when it's satisfied ... in a "finally" clause, of course....
        > > >
        > >
        > > At which point it may no longer be the simplest thing that could
        > > possibly work ;-)
        > >
        > > How would you change the test so that it was?
        > >
        >
        >

        --
        Bill Michell
        billmichell@...






        [Non-text portions of this message have been removed]
      Your message has been successfully submitted and would be delivered to recipients shortly.