Re: [scrumdevelopment] Re: (Executable) Test Specification?
- Hi Mark,
I don't believe that asynchronous events are a smell. Just the
opposite, they're a critical fact of life. What I am saying is that
you should consider de-coupling the asynchronous message handling from
your business rules testing. Make each test atomic and with a specific
reason for existing. Test your network handing in network handling
tests, test your business logic in specific business logic testing,
Hope this clarifies,
On Mon, Sep 29, 2008 at 8:41 AM, woynam <woyna@...> wrote:
> Why do you believe asynchronous events are a smell? We've been burned
> many time in the past when we didn't verify specific aspects of the
> system state. Many of the events generated by the system are
> asynchronous in nature.
> For example, the fill report for an order, and the last sale and quote
> messages are all asynchronous. It's insufficient to say that a
> successful trade simply involves entering orders into the system
> without errors. If the last sale and book quote is not updating, the
> system is not working.
> While there is a time-dependent aspect of the test, it's not really a
> test of a non-functional requirement. Given the asynchronous nature of
> the data, the test tool has to determine if the data has arrived, or
> if the test should fail, and move on to the next test. Given that our
> test systems tend to be much slower than our production system, the
> values used are relatively high.
> --- In email@example.com, "Tim Walker" <walketim@...>
>> Hi Mark,
>> You asked a few questions. I will deal with them one by one.
>> "Now, there are 8 different operations that take place in this test,
>> each with a different set of parameters. In addition, some of the
>> checks require receiving callbacks, i.e. asynchronous messages
>> within N seconds. How would one set up this test using Fit or FitNesse?"
>> This would be a classic example of a Fit/FitNesse DoFixture
>> (FitLibrary) as you are testing a workflow. You have complicated the
>> test by specifying the asynchronous callback nature in this test,
>> which is kind of a smell. For this test I would mock the callback and
>> isolate the test of the business logic by making a synchronous call to
>> the mock object. I would then test the asynchronous message handling
>> in another test, or I would handle the asynchonous aspect in the
>> fixture itself by writing a facade to this functionality and
>> specifying the non-functional requirements like:
>> |Book Order Response Time < | 4 | Seconds |
>> The fixture method would have a signature of:
>> public boolean BookOrderResponseTimeLessThanSeconds (int maxSeconds)
>> "The simple table seems OK for input -> output type tests with a
>> single call, but how does one string them together, and pass the
> entire set
>> as an atomic operation?"
>> The DoFixture would be the way to express this. In the test below you
>> probably have 10-20 discrete tests being conducted. All of which would
>> need to pass to make the whole test pass. You would probably want to
>> do some negative testing as well by using the reject keyword to invert
>> the state of a test that *should* fail so that it passes. A DoFIxture
>> supports all other Fixture types. If the first table specifies a
>> DoFixture than a "Flow Object" is created that handles the rest of the
>> test, which could be any other type of test, for example, Row, Column
>> or Custom table handlers.
>> "How does one deal with time sensitive operations, especially ones
>> that deal with asynchronous deliver of data from the system to the
>> As mentioned the use of mock objects could isolate the business logic
>> from the asynchronous nature and then the asynchronous behavior could
>> be tested in a discrete set of test pages.
>> This is just one approach and it would be a decent starting place. As
>> you discovered better ways in your environment you'd refactor the
>> tests accordingly.
>> Please let me know if this is unclear or you have other thoughts. I'd
>> also suggest the Yahoo FitNesse forum as a great source of support for
>> questions about FIT/FitNesses.
>> Thanks very much,
>> On Fri, Sep 26, 2008 at 7:06 PM, aacockburn <acockburn@...> wrote:
>> > Thanks for the nice example ... let's turn the question around
>> > and see if anything emerges ...
>> > If you were going to write what's on your mind for that example
>> > in any economical format understandable by humans, without
>> > regard to whether it is executable, how would you write it?
>> > cheers - Alistair
>> > --- In firstname.lastname@example.org, "woynam" <woyna@> wrote:
>> >> I was never convinced that a decision table is all one needs to
>> >> construct a test, i.e. inputs and outputs. How does one construct
>> >> tests with multiple different calls?
>> >> For example, to test a limit order to limit order trade, I might
>> > have to:
>> >> 1. Log in User 1
>> >> 2. Log in User 2
>> >> 3. Subscribe for Last Sale Messages for IBM
>> >> 4. Subscribe for Quote for IBM
>> >> 5. Verify that the IBM order book is empty by checking that the
>> >> current quote is $0.00 - $0.00 (Quote Query)
>> >> 6. Enter an IBM sell order for 20 @ $1.20 for User 1
>> >> 7. Verify the IBM quote, $0.00 - $1.20 (asynchronous callback)
>> >> 8. Enter an IBM buy order for 10 @ $1.20 for User 2
>> >> 9. Verify that User 1 receives an (asynchronous) fill report for 10
>> > @
>> >> $1.20
>> >> 10. 7. Verify that User 2 receives an (asynchronous) fill report for
>> >> 10 @ $1.20
>> >> 11. Verify the IBM quote, $0.00 - $1.20 (asynchronous callback)
>> >> 12. Verify that an IBM last sale was received (asynchronously) for
>> > 10
>> >> @ $1.20
>> >> 13. Log out User 1
>> >> 14. Log out User 2
>> >> Now, there are 8 different operations that take place in this test,
>> >> each with a different set of parameters. In addition, some of the
>> >> checks require receiving callbacks, i.e. asynchronous messages
>> > within
>> >> N seconds. How would one set up this test using Fit or FitNesse?
>> >> The simple table seems OK for input -> output type tests with a
>> > single
>> >> call, but how does one string them together, and pass the entire set
>> >> as an atomic operation? How does one deal with time sensitive
>> >> operations, especially ones that deal with asynchronous deliver of
>> >> data from the system to the user?
>> >> Mark
- God I've seen those disturbances too, but like I said, I respect both
perspectives. It's a very fine line.... very fine line....
Someday we'll figure out a way to punish these offenders and abusers. :-)
--- In email@example.com, "aacockburn" <acockburn@...>
> Hi, Chuck --- not much there for me to disagree with, which is
> remarkable considering how long it is :-)
> -- except possibly that I have managed to live long
> enough (10 years :) to see such seeming minor phrases become
> major disturbances that never get put out. I was sensing
> 'agile specification' as one of these, and hence jumped
> (seemingly overenergetically) right away.
> I wish others would have your sense of pragmatism,
> (Oddly enough, the use case book seemingly achingly Shu level
> to me - I can't imagine writing more Shu level. ouch. :)
> (for Ri level, see my Agile Software Development book, or even
> the Crystal Clear book, which is a Ha-Ri mix.)
> --- In firstname.lastname@example.org, "chuckspublicprofile" <chuck-
> lists2@> wrote:
> > (Forgive me if I misuse the Shu-Ha-Ri thing -- just recently read it
> > for the first time and since Mr. Cockburn is part of the primary
> > audience in this discussion, wanted to try to connect with his
> > communication patterns.)
> > I see value in both perspectives, so let me say that first. I'm the
> > OP, and I wasn't looking to quote anyone so much as understanding
> > this "interesting facet of agile" was all about, and I had trouble
> > finding it defined elsewhere.
> > I think I could make a case for saying that a) The AS should be a
> > strongly suggested or required artifact/activity but b) only in
> > part because the definition is still pretty wide open. However, I
> > could care less whether it should be so, as every client I've worked
> > at evaluates said tools and chooses them based on their own needs.
> > such, I have no compelling reason to argue the AS, or the name for
> > that matter.
> > Now, back to "providing problem context" and truth in advertising,
> > I think more defined/practical/Shu (whatever you want to call it)
> > processes/techniques/activities have value. I also think that more
> > qualified/contexted/Ri (less defined, more general, etc)
> > processes(etc) have value too.
> > One could argue it both ways. For instance, I have been a huge fan
> > Mr. Cockburns work on Use Cases. However, he wrote one book (I
> > it was Writing Effective Use Cases), that totally threw off one of
> > clients. The book was so advanced (Ha-Ri?), so qualified, so
> > and context sensitive, that these clients of mine were confushed and
> > mis-using his advice. They were writing use cases, that within a
> > SINGLE use case, they were communicating at vary high ranges of
> > altitude (cloud-sea, kite-underwater, etc). That was just one of
> > misuses they were committing. They were operating at the Shu level
> > but thinking they were at the Ri level (this happens a LOT in the
> > industry, IMO). It was a mess. Having had 8 years of prior Use
> > experience and study of Mr. Cockburn's works, I showed up and
> > to get them to focus on sea level as a first step, and things got
> > markedly better. I think Mr. Cockburn even says, in that book, that
> > sea level is where you tend to get a lot of ROI.
> > My point in telling this story is as follows:
> > - Having both very practical(Shu) level advice and very highly
> > "context sensitive"(Ri?) advice is good. Even practical level
> > that has very little context can be good. In the end, people solve
> > problems, not books/articles/emails. As such, both ends of the
> > spectrum can get people in trouble.
> > - People can misuse BOTH levels very easily, quoting Mr. Agile Guru
> > much as they want. Anyone who believes something solely because Mr.
> > Agile said it is a fool anyway, and I don't actually know many
> > who do it -- as most developers thing their ideas are just as good
> > better than any Gurus(which I disagree with in general)...
> > And full circle, back to the AS. First, it's a very remote point,
> > we saw -- it's barely mentioned anywhere on the internet, and not in
> > any agile books that I'm aware of. Mr. Sutherland likes it and has
> > been very careful to define the term loosely. AFAIK, Mr. Sutherland
> > hasn't mentioned anywhere that it is a required activity/artifact,
> > other than to gain a measly 2-3 points on the enhanced Nokia test,
> > in his training courses), something only a small fraction of the
> > industry pays attention to anyway.
> > In conclusion, I don't think, currently, that the use of the term AS
> > is really going to have much impact and certainly much negative
> > impact. Hopefully, when Mr. Sutherland gets time, he can expound a
> > little more on it. In the meantime, I get what he is saying, and
> > not going to go tell anyone that we have to do it because Mr. Agile
> > Guru X said it. Some dumbarses might do that, but really.... there
> > NO cure for the dumbarse disease, so we might as well realize that
> > move on. Just my 1 cent. (I could charge 2 cents if I was
> a "guru" --
> > hee hee)
> > Charles Bradley
> > --- In email@example.com, "aacockburn" <acockburn@>
> > wrote:
> > >
> > > --- In firstname.lastname@example.org, Robert Biddle
> > > <Robert_Biddle@> wrote:
> > > >
> > > > Oh, I did see and understand the objection.
> > > >
> > > > It's just that I didn't understand the strength with
> > > > which you objected. I still don't.
> > > >
> > > > Maybe I missed where it was being implied that everyone
> > > > should do the practice?
> > > > Or maybe I don't understand your feelings about the use of the
> > > > word "Agile"?
> > >
> > > No - it's not the word 'agile'.
> > >
> > > There are some people who hold a lot of influence in this
> > > Jeff's one. (OK, I'm one, and Ron's one, and Martin Fowler's one,
> > > Mary Poppendieck's one, etc.).
> > >
> > > When one of us makes a sweeping generalization that x-and-so is
> > > the "agile missing piece", then there will be lots and lots and
> > > lots of people who will run around quoting that person (as we
> > > already saw with Jeff's 'agile specification').
> > >
> > > And what they will be running around with will be damagingly
> > > wrong, because they are really citing "Mr. (Mrs.) So-and-So's
> > > Personally Used Possibly One-Off Solution to a Particular