Loading ...
Sorry, an error occurred while loading the content.

Re: [scrumdevelopment] Re: (Executable) Test Specification?

Expand Messages
  • Tim Walker
    Hi Mark, I don t believe that asynchronous events are a smell. Just the opposite, they re a critical fact of life. What I am saying is that you should consider
    Message 1 of 54 , Sep 29, 2008
    • 0 Attachment
      Hi Mark,

      I don't believe that asynchronous events are a smell. Just the
      opposite, they're a critical fact of life. What I am saying is that
      you should consider de-coupling the asynchronous message handling from
      your business rules testing. Make each test atomic and with a specific
      reason for existing. Test your network handing in network handling
      tests, test your business logic in specific business logic testing,
      etc.

      Hope this clarifies,

      Tim

      On Mon, Sep 29, 2008 at 8:41 AM, woynam <woyna@...> wrote:
      >
      > Thanks.
      >
      > Why do you believe asynchronous events are a smell? We've been burned
      > many time in the past when we didn't verify specific aspects of the
      > system state. Many of the events generated by the system are
      > asynchronous in nature.
      >
      > For example, the fill report for an order, and the last sale and quote
      > messages are all asynchronous. It's insufficient to say that a
      > successful trade simply involves entering orders into the system
      > without errors. If the last sale and book quote is not updating, the
      > system is not working.
      >
      > While there is a time-dependent aspect of the test, it's not really a
      > test of a non-functional requirement. Given the asynchronous nature of
      > the data, the test tool has to determine if the data has arrived, or
      > if the test should fail, and move on to the next test. Given that our
      > test systems tend to be much slower than our production system, the
      > values used are relatively high.
      >
      > Mark
      >
      > --- In scrumdevelopment@yahoogroups.com, "Tim Walker" <walketim@...>
      > wrote:
      >
      >>
      >> Hi Mark,
      >>
      >> You asked a few questions. I will deal with them one by one.
      >>
      >> "Now, there are 8 different operations that take place in this test,
      >> each with a different set of parameters. In addition, some of the
      >> checks require receiving callbacks, i.e. asynchronous messages
      >> within N seconds. How would one set up this test using Fit or FitNesse?"
      >>
      >> This would be a classic example of a Fit/FitNesse DoFixture
      >> (FitLibrary) as you are testing a workflow. You have complicated the
      >> test by specifying the asynchronous callback nature in this test,
      >> which is kind of a smell. For this test I would mock the callback and
      >> isolate the test of the business logic by making a synchronous call to
      >> the mock object. I would then test the asynchronous message handling
      >> in another test, or I would handle the asynchonous aspect in the
      >> fixture itself by writing a facade to this functionality and
      >> specifying the non-functional requirements like:
      >>
      >> |Book Order Response Time < | 4 | Seconds |
      >>
      >> The fixture method would have a signature of:
      >>
      >> public boolean BookOrderResponseTimeLessThanSeconds (int maxSeconds)
      >>
      >> "The simple table seems OK for input -> output type tests with a
      >> single call, but how does one string them together, and pass the
      > entire set
      >> as an atomic operation?"
      >>
      >> The DoFixture would be the way to express this. In the test below you
      >> probably have 10-20 discrete tests being conducted. All of which would
      >> need to pass to make the whole test pass. You would probably want to
      >> do some negative testing as well by using the reject keyword to invert
      >> the state of a test that *should* fail so that it passes. A DoFIxture
      >> supports all other Fixture types. If the first table specifies a
      >> DoFixture than a "Flow Object" is created that handles the rest of the
      >> test, which could be any other type of test, for example, Row, Column
      >> or Custom table handlers.
      >>
      >> "How does one deal with time sensitive operations, especially ones
      >> that deal with asynchronous deliver of data from the system to the
      >> user?"
      >>
      >> As mentioned the use of mock objects could isolate the business logic
      >> from the asynchronous nature and then the asynchronous behavior could
      >> be tested in a discrete set of test pages.
      >>
      >> This is just one approach and it would be a decent starting place. As
      >> you discovered better ways in your environment you'd refactor the
      >> tests accordingly.
      >>
      >> Please let me know if this is unclear or you have other thoughts. I'd
      >> also suggest the Yahoo FitNesse forum as a great source of support for
      >> questions about FIT/FitNesses.
      >>
      >> Thanks very much,
      >>
      >> Tim
      >>
      >>
      >> On Fri, Sep 26, 2008 at 7:06 PM, aacockburn <acockburn@...> wrote:
      >> > Thanks for the nice example ... let's turn the question around
      >> > and see if anything emerges ...
      >> >
      >> > If you were going to write what's on your mind for that example
      >> > in any economical format understandable by humans, without
      >> > regard to whether it is executable, how would you write it?
      >> >
      >> > cheers - Alistair
      >> >
      >> > --- In scrumdevelopment@yahoogroups.com, "woynam" <woyna@> wrote:
      >> >>
      >> >>
      >> >> I was never convinced that a decision table is all one needs to
      >> >> construct a test, i.e. inputs and outputs. How does one construct
      >> >> tests with multiple different calls?
      >> >>
      >> >> For example, to test a limit order to limit order trade, I might
      >> > have to:
      >> >>
      >> >> 1. Log in User 1
      >> >> 2. Log in User 2
      >> >> 3. Subscribe for Last Sale Messages for IBM
      >> >> 4. Subscribe for Quote for IBM
      >> >> 5. Verify that the IBM order book is empty by checking that the
      >> >> current quote is $0.00 - $0.00 (Quote Query)
      >> >> 6. Enter an IBM sell order for 20 @ $1.20 for User 1
      >> >> 7. Verify the IBM quote, $0.00 - $1.20 (asynchronous callback)
      >> >> 8. Enter an IBM buy order for 10 @ $1.20 for User 2
      >> >> 9. Verify that User 1 receives an (asynchronous) fill report for 10
      >> > @
      >> >> $1.20
      >> >> 10. 7. Verify that User 2 receives an (asynchronous) fill report for
      >> >> 10 @ $1.20
      >> >> 11. Verify the IBM quote, $0.00 - $1.20 (asynchronous callback)
      >> >> 12. Verify that an IBM last sale was received (asynchronously) for
      >> > 10
      >> >> @ $1.20
      >> >> 13. Log out User 1
      >> >> 14. Log out User 2
      >> >>
      >> >>
      >> >> Now, there are 8 different operations that take place in this test,
      >> >> each with a different set of parameters. In addition, some of the
      >> >> checks require receiving callbacks, i.e. asynchronous messages
      >> > within
      >> >> N seconds. How would one set up this test using Fit or FitNesse?
      >> >>
      >> >> The simple table seems OK for input -> output type tests with a
      >> > single
      >> >> call, but how does one string them together, and pass the entire set
      >> >> as an atomic operation? How does one deal with time sensitive
      >> >> operations, especially ones that deal with asynchronous deliver of
      >> >> data from the system to the user?
      >> >>
      >> >> Mark
      >> >>
      >> >>
      >> >
      >> >
      >>
      >
      >
    • chuckspublicprofile
      God I ve seen those disturbances too, but like I said, I respect both perspectives. It s a very fine line.... very fine line.... Someday we ll figure out a
      Message 54 of 54 , Oct 6, 2008
      • 0 Attachment
        God I've seen those disturbances too, but like I said, I respect both
        perspectives. It's a very fine line.... very fine line....

        Someday we'll figure out a way to punish these offenders and abusers. :-)

        Charles



        --- In scrumdevelopment@yahoogroups.com, "aacockburn" <acockburn@...>
        wrote:
        >
        > Hi, Chuck --- not much there for me to disagree with, which is
        > remarkable considering how long it is :-)
        >
        > -- except possibly that I have managed to live long
        > enough (10 years :) to see such seeming minor phrases become
        > major disturbances that never get put out. I was sensing
        > 'agile specification' as one of these, and hence jumped
        > (seemingly overenergetically) right away.
        >
        > I wish others would have your sense of pragmatism,
        >
        > (Oddly enough, the use case book seemingly achingly Shu level
        > to me - I can't imagine writing more Shu level. ouch. :)
        > (for Ri level, see my Agile Software Development book, or even
        > the Crystal Clear book, which is a Ha-Ri mix.)
        >
        > Alistair
        >
        > --- In scrumdevelopment@yahoogroups.com, "chuckspublicprofile" <chuck-
        > lists2@> wrote:
        > >
        > >
        > > (Forgive me if I misuse the Shu-Ha-Ri thing -- just recently read it
        > > for the first time and since Mr. Cockburn is part of the primary
        > > audience in this discussion, wanted to try to connect with his
        > > communication patterns.)
        > >
        > > I see value in both perspectives, so let me say that first. I'm the
        > > OP, and I wasn't looking to quote anyone so much as understanding
        > what
        > > this "interesting facet of agile" was all about, and I had trouble
        > > finding it defined elsewhere.
        > >
        > > I think I could make a case for saying that a) The AS should be a
        > > strongly suggested or required artifact/activity but b) only in
        > large
        > > part because the definition is still pretty wide open. However, I
        > > could care less whether it should be so, as every client I've worked
        > > at evaluates said tools and chooses them based on their own needs.
        > As
        > > such, I have no compelling reason to argue the AS, or the name for
        > > that matter.
        > >
        > > Now, back to "providing problem context" and truth in advertising,
        > etc...
        > >
        > > I think more defined/practical/Shu (whatever you want to call it)
        > > processes/techniques/activities have value. I also think that more
        > > qualified/contexted/Ri (less defined, more general, etc)
        > > processes(etc) have value too.
        > >
        > > One could argue it both ways. For instance, I have been a huge fan
        > of
        > > Mr. Cockburns work on Use Cases. However, he wrote one book (I
        > think
        > > it was Writing Effective Use Cases), that totally threw off one of
        > my
        > > clients. The book was so advanced (Ha-Ri?), so qualified, so
        > detailed
        > > and context sensitive, that these clients of mine were confushed and
        > > mis-using his advice. They were writing use cases, that within a
        > > SINGLE use case, they were communicating at vary high ranges of
        > > altitude (cloud-sea, kite-underwater, etc). That was just one of
        > many
        > > misuses they were committing. They were operating at the Shu level
        > > but thinking they were at the Ri level (this happens a LOT in the
        > > industry, IMO). It was a mess. Having had 8 years of prior Use
        > Case
        > > experience and study of Mr. Cockburn's works, I showed up and
        > managed
        > > to get them to focus on sea level as a first step, and things got
        > > markedly better. I think Mr. Cockburn even says, in that book, that
        > > sea level is where you tend to get a lot of ROI.
        > >
        > > My point in telling this story is as follows:
        > >
        > > - Having both very practical(Shu) level advice and very highly
        > > "context sensitive"(Ri?) advice is good. Even practical level
        > advice
        > > that has very little context can be good. In the end, people solve
        > > problems, not books/articles/emails. As such, both ends of the
        > > spectrum can get people in trouble.
        > >
        > > - People can misuse BOTH levels very easily, quoting Mr. Agile Guru
        > as
        > > much as they want. Anyone who believes something solely because Mr.
        > > Agile said it is a fool anyway, and I don't actually know many
        > people
        > > who do it -- as most developers thing their ideas are just as good
        > or
        > > better than any Gurus(which I disagree with in general)...
        > >
        > > And full circle, back to the AS. First, it's a very remote point,
        > as
        > > we saw -- it's barely mentioned anywhere on the internet, and not in
        > > any agile books that I'm aware of. Mr. Sutherland likes it and has
        > > been very careful to define the term loosely. AFAIK, Mr. Sutherland
        > > hasn't mentioned anywhere that it is a required activity/artifact,
        > > other than to gain a measly 2-3 points on the enhanced Nokia test,
        > (or
        > > in his training courses), something only a small fraction of the
        > > industry pays attention to anyway.
        > >
        > > In conclusion, I don't think, currently, that the use of the term AS
        > > is really going to have much impact and certainly much negative
        > > impact. Hopefully, when Mr. Sutherland gets time, he can expound a
        > > little more on it. In the meantime, I get what he is saying, and
        > I'm
        > > not going to go tell anyone that we have to do it because Mr. Agile
        > > Guru X said it. Some dumbarses might do that, but really.... there
        > is
        > > NO cure for the dumbarse disease, so we might as well realize that
        > and
        > > move on. Just my 1 cent. (I could charge 2 cents if I was
        > a "guru" --
        > > hee hee)
        > >
        > > Charles Bradley
        > >
        > >
        > > --- In scrumdevelopment@yahoogroups.com, "aacockburn" <acockburn@>
        > > wrote:
        > > >
        > > > --- In scrumdevelopment@yahoogroups.com, Robert Biddle
        > > > <Robert_Biddle@> wrote:
        > > > >
        > > > > Oh, I did see and understand the objection.
        > > > >
        > > > > It's just that I didn't understand the strength with
        > > > > which you objected. I still don't.
        > > > >
        > > > > Maybe I missed where it was being implied that everyone
        > > > > should do the practice?
        > > > > Or maybe I don't understand your feelings about the use of the
        > > > > word "Agile"?
        > > >
        > > > No - it's not the word 'agile'.
        > > >
        > > > There are some people who hold a lot of influence in this
        > industry.
        > > > Jeff's one. (OK, I'm one, and Ron's one, and Martin Fowler's one,
        > and
        > > > Mary Poppendieck's one, etc.).
        > > >
        > > > When one of us makes a sweeping generalization that x-and-so is
        > > > the "agile missing piece", then there will be lots and lots and
        > > > lots of people who will run around quoting that person (as we
        > > > already saw with Jeff's 'agile specification').
        > > >
        > > > And what they will be running around with will be damagingly
        > > > wrong, because they are really citing "Mr. (Mrs.) So-and-So's
        > > > Personally Used Possibly One-Off Solution to a Particular
        > >
        >
      Your message has been successfully submitted and would be delivered to recipients shortly.