Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Examples of good acceptance test criteria

Expand Messages
  • Wouter Lagerweij
    Hi Jack, I rather like Ken Pugh s book ( http://www.amazon.com/Lean-Agile-Acceptance-Test-Driven-Development-Collaboration/dp/0321714083 ) It s very hands
    Message 1 of 17 , Jul 22 12:38 PM
    • 0 Attachment
      Hi Jack,

      I rather like Ken Pugh's book (
      http://www.amazon.com/Lean-Agile-Acceptance-Test-Driven-Development-Collaboration/dp/0321714083
      )
      It's very 'hands on', describing interactions between team and PO, and
      giving many concrete examples using an example project.

      The Specification by Example book (Gojko Adzic) is also very good, but is
      more focused on the process, in my opinion.

      Wouter


      On Mon, Jul 22, 2013 at 9:16 PM, JackM <jack@...> wrote:

      > **
      >
      >
      > I am looking for really good examples of acceptance test criteria so that
      > I can help the PO's and Teams to understand what makes really good
      > acceptance test criteria.
      >
      > After many sprints I am still getting the "Don't have clarity of what is
      > required". Acceptance tests are too high level.
      >
      > I have read through Mike's user stories applied book and their are
      > examples there but they're rather simple in nature for a basic website.
      >
      > If you can point me to any literature, blog etc on this topic i'd
      > sincerely appreciate it.
      >
      > Thanks
      > jack
      >
      >
      >



      --
      Wouter Lagerweij | wouter@...
      http://www.lagerweij.com | @wouterla <http://twitter.com/#!/wouterla>


      [Non-text portions of this message have been removed]
    • Ron Jeffries
      Hi Jack, ... In addition to the reading suggested already, how about you tell us some of your user stories and we ll brainstorm some tests. Ron Jeffries
      Message 2 of 17 , Jul 22 1:09 PM
      • 0 Attachment
        Hi Jack,

        On Jul 22, 2013, at 3:16 PM, "JackM" <jack@...> wrote:

        > I am looking for really good examples of acceptance test criteria so that I can help the PO's and Teams to understand what makes really good acceptance test criteria.


        In addition to the reading suggested already, how about you tell us some of your user stories and we'll brainstorm some tests.

        Ron Jeffries
        www.XProgramming.com
        If it is more than you need, it is waste. -- Andy Seidl



        [Non-text portions of this message have been removed]
      • Adam Sroka
        I second those recommendations (https://leanpub.com/cucumber_and_cheese and http://pragprog.com/book/hwcuc/the-cucumber-book) Note that while Cucumber is an
        Message 3 of 17 , Jul 22 1:13 PM
        • 0 Attachment
          I second those recommendations (https://leanpub.com/cucumber_and_cheese and
          http://pragprog.com/book/hwcuc/the-cucumber-book)

          Note that while Cucumber is an excellent tool compatible with many
          different programming languages you don't need to be using Cucumber to
          benefit from writing your scenarios this way. There are also alternative
          tools that use similar syntax.



          On Mon, Jul 22, 2013 at 3:37 PM, Francis Fish <francis@...>wrote:

          > **
          >
          >
          > On Mon, Jul 22, 2013 at 8:16 PM, JackM <jack@...> wrote:
          >
          > > **
          > >
          > >
          > > I am looking for really good examples of acceptance test criteria so that
          > > I can help the PO's and Teams to understand what makes really good
          > > acceptance test criteria.
          > >
          > > After many sprints I am still getting the "Don't have clarity of what is
          > > required". Acceptance tests are too high level.
          > >
          > > I have read through Mike's user stories applied book and their are
          > > examples there but they're rather simple in nature for a basic website.
          > >
          > > If you can point me to any literature, blog etc on this topic i'd
          > > sincerely appreciate it.
          > >
          > > Thanks
          > > jack
          > >
          >
          > I think that if you were doing outside in dev, as described in the Cucumber
          > book, and also the leanpub book Cucumber and Cheese, then the lower level
          > tests should come out as part of the process, as part of the dialogue
          > between the test and dev teams.
          >
          > The criteria come out of the dialogue, it sounds like the people involved
          > need to work together more and understand what 'clean flow' for the next
          > people in the process means, this is a classic example of waste in fact.
          >
          > Thanks and regards,
          >
          > Francis
          >
          > Unicorns in the Mist https://leanpub.com/unicorns
          > Follow me on twitter https://twitter.com/fjfish
          > Blog at http://www.francisfish.com
          > Books at https://leanpub.com/u/fjfish
          > CV http://www.pharmarketeer.com/francis.html
          >
          > I have no intention of apologizing for believing in people, for insisting
          > that we all use this moment and these assets to create some art and improve
          > the world around us.
          > To do anything less than that is a crime. - Seth Godin
          >
          >
          > [Non-text portions of this message have been removed]
          >
          >
          >


          [Non-text portions of this message have been removed]
        • Lior Friedman
          ... I can help the PO s and Teams to understand what makes really good acceptance test criteria. ... required . Acceptance tests are too high level. ... How
          Message 4 of 17 , Jul 22 1:22 PM
          • 0 Attachment
            On Jul 22, 2013 10:16 PM, "JackM" <jack@...> wrote:
            >
            >
            >
            > I am looking for really good examples of acceptance test criteria so that
            I can help the PO's and Teams to understand what makes really good
            acceptance test criteria.
            >
            > After many sprints I am still getting the "Don't have clarity of what is
            required". Acceptance tests are too high level.
            >

            How about putting one of your stories as an example, so we can have
            something to work on?

            Lior


            [Non-text portions of this message have been removed]
          • Ron Jeffries
            Lior, ... Brilliant! :) Ron Jeffries www.XProgramming.com Sometimes you just have to stop holding on with both hands, both feet, and your tail, to get
            Message 5 of 17 , Jul 22 1:25 PM
            • 0 Attachment
              Lior,

              On Jul 22, 2013, at 4:22 PM, Lior Friedman <lfriedmal@...> wrote:

              > How about putting one of your stories as an example, so we can have
              > something to work on?


              Brilliant! :)

              Ron Jeffries
              www.XProgramming.com
              Sometimes you just have to stop holding on with both hands, both feet, and your tail, to get someplace better.
              Of course you might plummet to the earth and die, but probably not: you were made for this.



              [Non-text portions of this message have been removed]
            • Phlip
              ... When I wrote Morelia viridis (a Python clone for Cucumber, in like 5% of the lines of code), I was not working with an XP team. But the only point of
              Message 6 of 17 , Jul 22 5:44 PM
              • 0 Attachment
                > Note that while Cucumber is an excellent tool compatible with many
                > different programming languages you don't need to be using Cucumber to
                > benefit from writing your scenarios this way. There are also alternative
                > tools that use similar syntax.

                When I wrote Morelia viridis (a Python clone for Cucumber, in like 5%
                of the lines of code), I was not working with an XP team. But the only
                point of Cucumber notation ("Gherkin") is to let an onsite customer
                read and write the scenarios as business rules, without the extra
                clutter that a real programming language drags in.

                But one of my assignments went something like this: "If the customer
                orders up to 4 tee-shirts and one speed-suit, stick them all in a
                one-wet-suit box. But if they order two speed-suits, put these into a
                wet-suit box. If they order two speed-suits and a wet-suit, put them
                in a two-wet-suit box." And so on.

                I wrote all that up as Cucumber scenarios, passed them, and e-mailed them out.

                Our client emailed back one of the scenarios emended (4 tee-shirts & 1
                wet-suit in a single wet-suit box), and I entered that and passed it.

                _that_ is what Cucumber is for. And that's why DHH wrote a post
                flaming Cucumber, because he's tired of seeing everyone use it wrong.
              • Adam Sroka
                I m not sure how this is a response to what I wrote. If your point is that the customer needs to be the one driving the scenarios then you are preaching to the
                Message 7 of 17 , Jul 22 6:21 PM
                • 0 Attachment
                  I'm not sure how this is a response to what I wrote. If your point is that
                  the customer needs to be the one driving the scenarios then you are
                  preaching to the choir. Although, I would never email them the scenarios. I
                  would insist on a face-to-face conversation.

                  I would never use Cucumber to write tests for myself or my fellow
                  programmers. I would use something like RSpec or xUnit for that. Cucumber
                  is for facilitating conversations with customers. If the customer is not
                  involved then it is the wrong tool (And quite likely the wrong project.)


                  On Mon, Jul 22, 2013 at 8:44 PM, Phlip <phlip2005@...> wrote:

                  > **
                  >
                  >
                  > > Note that while Cucumber is an excellent tool compatible with many
                  > > different programming languages you don't need to be using Cucumber to
                  > > benefit from writing your scenarios this way. There are also alternative
                  > > tools that use similar syntax.
                  >
                  > When I wrote Morelia viridis (a Python clone for Cucumber, in like 5%
                  > of the lines of code), I was not working with an XP team. But the only
                  > point of Cucumber notation ("Gherkin") is to let an onsite customer
                  > read and write the scenarios as business rules, without the extra
                  > clutter that a real programming language drags in.
                  >
                  > But one of my assignments went something like this: "If the customer
                  > orders up to 4 tee-shirts and one speed-suit, stick them all in a
                  > one-wet-suit box. But if they order two speed-suits, put these into a
                  > wet-suit box. If they order two speed-suits and a wet-suit, put them
                  > in a two-wet-suit box." And so on.
                  >
                  > I wrote all that up as Cucumber scenarios, passed them, and e-mailed them
                  > out.
                  >
                  > Our client emailed back one of the scenarios emended (4 tee-shirts & 1
                  > wet-suit in a single wet-suit box), and I entered that and passed it.
                  >
                  > _that_ is what Cucumber is for. And that's why DHH wrote a post
                  > flaming Cucumber, because he's tired of seeing everyone use it wrong.
                  >
                  >
                  >


                  [Non-text portions of this message have been removed]
                • Phlip
                  ... Blame bad editing from a roadside stop at a Starbucks. I answered the original question what s good acceptance test criteria .
                  Message 8 of 17 , Jul 22 10:49 PM
                  • 0 Attachment
                    > I'm not sure how this is a response to what I wrote.

                    Blame bad editing from a roadside stop at a Starbucks. I answered the
                    original question "what's good acceptance test criteria".
                  • George Dinwiddie
                    Adam, ... I would. Sometimes I need to facilitate the conversation with myself. I find that expressing the business problem in english is often better for
                    Message 9 of 17 , Jul 23 7:55 AM
                    • 0 Attachment
                      Adam,

                      On 7/22/13 9:21 PM, Adam Sroka wrote:
                      > I'm not sure how this is a response to what I wrote. If your point is that
                      > the customer needs to be the one driving the scenarios then you are
                      > preaching to the choir. Although, I would never email them the scenarios. I
                      > would insist on a face-to-face conversation.
                      >
                      > I would never use Cucumber to write tests for myself or my fellow
                      > programmers. I would use something like RSpec or xUnit for that. Cucumber
                      > is for facilitating conversations with customers. If the customer is not
                      > involved then it is the wrong tool (And quite likely the wrong project.)

                      I would. Sometimes I need to facilitate the conversation with myself. I
                      find that expressing the business problem in "english" is often better
                      for that than expressing it in code.

                      If the Customer is not involved, that's a different problem. Getting the
                      Customer involved may take more than just a BDD/ATDD tool. ;-)

                      - George

                      P.S. Nice examples, Phlip.

                      >
                      >
                      > On Mon, Jul 22, 2013 at 8:44 PM, Phlip <phlip2005@...> wrote:
                      >
                      >> **
                      >>
                      >>
                      >>> Note that while Cucumber is an excellent tool compatible with many
                      >>> different programming languages you don't need to be using Cucumber to
                      >>> benefit from writing your scenarios this way. There are also alternative
                      >>> tools that use similar syntax.
                      >>
                      >> When I wrote Morelia viridis (a Python clone for Cucumber, in like 5%
                      >> of the lines of code), I was not working with an XP team. But the only
                      >> point of Cucumber notation ("Gherkin") is to let an onsite customer
                      >> read and write the scenarios as business rules, without the extra
                      >> clutter that a real programming language drags in.
                      >>
                      >> But one of my assignments went something like this: "If the customer
                      >> orders up to 4 tee-shirts and one speed-suit, stick them all in a
                      >> one-wet-suit box. But if they order two speed-suits, put these into a
                      >> wet-suit box. If they order two speed-suits and a wet-suit, put them
                      >> in a two-wet-suit box." And so on.
                      >>
                      >> I wrote all that up as Cucumber scenarios, passed them, and e-mailed them
                      >> out.
                      >>
                      >> Our client emailed back one of the scenarios emended (4 tee-shirts & 1
                      >> wet-suit in a single wet-suit box), and I entered that and passed it.
                      >>
                      >> _that_ is what Cucumber is for. And that's why DHH wrote a post
                      >> flaming Cucumber, because he's tired of seeing everyone use it wrong.

                      --
                      Want to speak at AgileDC October 8, 2013? http://agiledc.org/speak/
                      ----------------------------------------------------------------------
                      * George Dinwiddie * http://blog.gdinwiddie.com
                      Software Development http://www.idiacomputing.com
                      Consultant and Coach http://www.agilemaryland.org
                      ----------------------------------------------------------------------
                    • Adam Sroka
                      On Tue, Jul 23, 2013 at 10:55 AM, George Dinwiddie ... Both are valid choices, and I have a lot of other friends who would do that too. I am a bit weary of
                      Message 10 of 17 , Jul 23 8:37 AM
                      • 0 Attachment
                        On Tue, Jul 23, 2013 at 10:55 AM, George Dinwiddie
                        <lists@...>wrote:

                        > **
                        >
                        >
                        > Adam,
                        >
                        >
                        > On 7/22/13 9:21 PM, Adam Sroka wrote:
                        > > I'm not sure how this is a response to what I wrote. If your point is
                        > that
                        > > the customer needs to be the one driving the scenarios then you are
                        > > preaching to the choir. Although, I would never email them the
                        > scenarios. I
                        > > would insist on a face-to-face conversation.
                        > >
                        > > I would never use Cucumber to write tests for myself or my fellow
                        > > programmers. I would use something like RSpec or xUnit for that. Cucumber
                        > > is for facilitating conversations with customers. If the customer is not
                        > > involved then it is the wrong tool (And quite likely the wrong project.)
                        >
                        > I would. Sometimes I need to facilitate the conversation with myself. I
                        > find that expressing the business problem in "english" is often better
                        > for that than expressing it in code.
                        >
                        >
                        Both are valid choices, and I have a lot of other friends who would do that
                        too. I am a bit weary of Cucumber for at least three reasons: 1) it adds
                        additional complexity to my test suite, 2) as the system grows in
                        complexity step definitions can collide and this takes additional effort to
                        resolve, 3) there is no guarantee that step definitions actually mean what
                        the english they are matching to suggests they mean and this takes
                        additional effort to ensure.

                        I am willing to accept the extra work implied by those issues if my
                        customer is willing to invest effort helping to write and maintain the
                        Gherkin. If they aren't then I will just avoid them by using a tool that is
                        closer to the code. RSpec is pretty darn close to english, and I can get
                        pretty close with xUnit as well (with a few exceptions, depending on the
                        specific tool.)


                        > If the Customer is not involved, that's a different problem. Getting the
                        > Customer involved may take more than just a BDD/ATDD tool. ;-)
                        >
                        >
                        Absolutely, and regardless of whether I am using Cucumber or not I am not
                        going to take money from a customer who isn't willing to be engaged in the
                        development of their product. Some customers need their hands held just to
                        write stories, others are willing to learn and maintain Gherkin, and there
                        are several levels in between.


                        [Non-text portions of this message have been removed]
                      • Steven Gordon
                        ... And there are many customers who will delegate the details of writing and maintaining Gherkin to non-programmer intermediaries like BAs or Testers. As long
                        Message 11 of 17 , Jul 23 9:14 AM
                        • 0 Attachment
                          On Tue, Jul 23, 2013 at 8:37 AM, Adam Sroka <adam.sroka@...> wrote:

                          > **
                          >
                          >
                          > On Tue, Jul 23, 2013 at 10:55 AM, George Dinwiddie
                          > <lists@...>wrote:
                          >
                          > > **
                          >
                          > >
                          > >
                          > > Adam,
                          > >
                          > >
                          > > On 7/22/13 9:21 PM, Adam Sroka wrote:
                          > > > I'm not sure how this is a response to what I wrote. If your point is
                          > > that
                          > > > the customer needs to be the one driving the scenarios then you are
                          > > > preaching to the choir. Although, I would never email them the
                          > > scenarios. I
                          > > > would insist on a face-to-face conversation.
                          > > >
                          > > > I would never use Cucumber to write tests for myself or my fellow
                          > > > programmers. I would use something like RSpec or xUnit for that.
                          > Cucumber
                          > > > is for facilitating conversations with customers. If the customer is
                          > not
                          > > > involved then it is the wrong tool (And quite likely the wrong
                          > project.)
                          > >
                          > > I would. Sometimes I need to facilitate the conversation with myself. I
                          > > find that expressing the business problem in "english" is often better
                          > > for that than expressing it in code.
                          > >
                          > >
                          > Both are valid choices, and I have a lot of other friends who would do that
                          > too. I am a bit weary of Cucumber for at least three reasons: 1) it adds
                          > additional complexity to my test suite, 2) as the system grows in
                          > complexity step definitions can collide and this takes additional effort to
                          > resolve, 3) there is no guarantee that step definitions actually mean what
                          > the english they are matching to suggests they mean and this takes
                          > additional effort to ensure.
                          >
                          > I am willing to accept the extra work implied by those issues if my
                          > customer is willing to invest effort helping to write and maintain the
                          > Gherkin. If they aren't then I will just avoid them by using a tool that is
                          > closer to the code. RSpec is pretty darn close to english, and I can get
                          > pretty close with xUnit as well (with a few exceptions, depending on the
                          > specific tool.)
                          >
                          >
                          > > If the Customer is not involved, that's a different problem. Getting the
                          > > Customer involved may take more than just a BDD/ATDD tool. ;-)
                          > >
                          > >
                          > Absolutely, and regardless of whether I am using Cucumber or not I am not
                          > going to take money from a customer who isn't willing to be engaged in the
                          > development of their product. Some customers need their hands held just to
                          > write stories, others are willing to learn and maintain Gherkin, and there
                          > are several levels in between.
                          >
                          >
                          And there are many customers who will delegate the details of writing and
                          maintaining Gherkin to non-programmer intermediaries like BAs or Testers.
                          As long as the customer and those intermediaries are active participants in
                          the project, especially in the conversation about each story, that can work
                          fine. Leveraging a language that non-programmers can read helps align
                          everybody's understanding.

                          SteveG


                          [Non-text portions of this message have been removed]
                        • Adam Sroka
                          ... Sure, as long as the BAs and/or testers are part of a whole team and not a distinct group that is physically separate. It is possible to teach testers to
                          Message 12 of 17 , Jul 23 9:29 AM
                          • 0 Attachment
                            On Tue, Jul 23, 2013 at 12:14 PM, Steven Gordon <sgordonphd@...>wrote:

                            > **
                            >
                            >
                            > On Tue, Jul 23, 2013 at 8:37 AM, Adam Sroka <adam.sroka@...> wrote:
                            > >
                            > >
                            > > On Tue, Jul 23, 2013 at 10:55 AM, George Dinwiddie
                            > > <lists@...>wrote:
                            > >
                            > >
                            > > > If the Customer is not involved, that's a different problem. Getting
                            > the
                            > > > Customer involved may take more than just a BDD/ATDD tool. ;-)
                            > > >
                            > > >
                            > > Absolutely, and regardless of whether I am using Cucumber or not I am not
                            > > going to take money from a customer who isn't willing to be engaged in
                            > the
                            > > development of their product. Some customers need their hands held just
                            > to
                            > > write stories, others are willing to learn and maintain Gherkin, and
                            > there
                            > > are several levels in between.
                            > >
                            > >
                            > And there are many customers who will delegate the details of writing and
                            > maintaining Gherkin to non-programmer intermediaries like BAs or Testers.
                            > As long as the customer and those intermediaries are active participants in
                            > the project, especially in the conversation about each story, that can work
                            > fine. Leveraging a language that non-programmers can read helps align
                            > everybody's understanding.
                            >
                            >
                            Sure, as long as the BAs and/or testers are part of a whole team and not a
                            distinct group that is physically separate.

                            It is possible to teach testers to write and maintain Gherkin. I have done
                            it. It is also possible to train BAs to act as effective customer proxies.
                            Neither is as good as the active participation of an actual onsite
                            customer, though.


                            [Non-text portions of this message have been removed]
                          • Adam Sroka
                            P.S. and, as you suggest, if both the BA/tester and the customer are active participants then that is even better, because BAs and testers bring skills to
                            Message 13 of 17 , Jul 23 9:32 AM
                            • 0 Attachment
                              P.S. and, as you suggest, if both the BA/tester and the customer are active
                              participants then that is even better, because BAs and testers bring skills
                              to writing the Gherkin that the customer may not have


                              On Tue, Jul 23, 2013 at 12:29 PM, Adam Sroka <adam.sroka@...> wrote:

                              >
                              >
                              >
                              > On Tue, Jul 23, 2013 at 12:14 PM, Steven Gordon <sgordonphd@...>wrote:
                              >
                              >> **
                              >>
                              >>
                              >> On Tue, Jul 23, 2013 at 8:37 AM, Adam Sroka <adam.sroka@...> wrote:
                              >> >
                              >> >
                              >> > On Tue, Jul 23, 2013 at 10:55 AM, George Dinwiddie
                              >> > <lists@...>wrote:
                              >> >
                              >> >
                              >>
                              >> > > If the Customer is not involved, that's a different problem. Getting
                              >> the
                              >> > > Customer involved may take more than just a BDD/ATDD tool. ;-)
                              >> > >
                              >> > >
                              >> > Absolutely, and regardless of whether I am using Cucumber or not I am
                              >> not
                              >> > going to take money from a customer who isn't willing to be engaged in
                              >> the
                              >> > development of their product. Some customers need their hands held just
                              >> to
                              >> > write stories, others are willing to learn and maintain Gherkin, and
                              >> there
                              >> > are several levels in between.
                              >> >
                              >> >
                              >> And there are many customers who will delegate the details of writing and
                              >> maintaining Gherkin to non-programmer intermediaries like BAs or Testers.
                              >> As long as the customer and those intermediaries are active participants
                              >> in
                              >> the project, especially in the conversation about each story, that can
                              >> work
                              >> fine. Leveraging a language that non-programmers can read helps align
                              >> everybody's understanding.
                              >>
                              >>
                              > Sure, as long as the BAs and/or testers are part of a whole team and not a
                              > distinct group that is physically separate.
                              >
                              > It is possible to teach testers to write and maintain Gherkin. I have done
                              > it. It is also possible to train BAs to act as effective customer proxies.
                              > Neither is as good as the active participation of an actual onsite
                              > customer, though.
                              >


                              [Non-text portions of this message have been removed]
                            • JackM
                              Thanks Ron, I ll see what I can pull together. Jack
                              Message 14 of 17 , Jul 26 10:44 AM
                              • 0 Attachment
                                Thanks Ron,

                                I'll see what I can pull together.

                                Jack

                                --- In extremeprogramming@yahoogroups.com, Ron Jeffries <ronjeffries@...> wrote:
                                >
                                > Hi Jack,
                                >
                                > On Jul 22, 2013, at 3:16 PM, "JackM" <jack@...> wrote:
                                >
                                > > I am looking for really good examples of acceptance test criteria so that I can help the PO's and Teams to understand what makes really good acceptance test criteria.
                                >
                                >
                                > In addition to the reading suggested already, how about you tell us some of your user stories and we'll brainstorm some tests.
                                >
                                > Ron Jeffries
                                > www.XProgramming.com
                                > If it is more than you need, it is waste. -- Andy Seidl
                                >
                                >
                                >
                                > [Non-text portions of this message have been removed]
                                >
                              Your message has been successfully submitted and would be delivered to recipients shortly.