Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Acceptance tests

Expand Messages
  • Kevin Smith
    ... I ll leave your questions for someone who has more AT experience than I do. ... Exactly correct. ... I suppose this is one side effect of UT s. It is
    Message 1 of 13 , Feb 15, 2001
    • 0 Attachment
      Haroon Sharif wrote:
      >PROBLEM:
      >I need to know the time of creation of Acceptance
      >Tests in XP.....?

      I'll leave your questions for someone who has
      more AT experience than I do.

      >And about Unit test .......correct me if i'm wrong
      >I know that Unit tests are created before u start
      >coding and even after you have done your coding ....
      >you keep updating the unit tests......and its done by
      >the Developers....

      Exactly correct.

      >they define the way points for the
      >developers so that they don't get carried away and
      >start coding thats not required.....YAGNI

      I suppose this is one side effect of UT's. It is
      definitely not one of the most important reasons
      to write UT's.

      Kevin
    • J. B. Rainsberger
      ... Our CTs are starting to look like this. I love it. I love refactoring. JBR.
      Message 2 of 13 , Nov 28, 2001
      • 0 Attachment
        Rob Myers wrote:

        >We had our Junit/HttpUnit test framework so beautifully well-factored that
        >the customer could have written the tests with very little help. They would
        >look something like this (from memory):
        >
        > goToPage(MyInformationPage.PAGE_NAME);
        > changeField(MyInformationPage.EMAIL_ADDRESS_FIELD,
        >"mynewaddress@...");
        > pressButton(MyInformationPage.SUBMIT_BUTTON);
        > assertNextPageIs(InformationAccepted.PAGE_NAME);
        > assertPageContainsText("mynewaddress@...");
        >
        >I'm already itching for a refactoring or two...
        >
        Our CTs are starting to look like this. I love it. I love refactoring.

        JBR.
      • Glen Stampoultzis
        ... to ... I keep my JSP s extremely thin (they are effectively just the view). I unit test the model and controller extensively using mock requests and
        Message 3 of 13 , Nov 28, 2001
        • 0 Attachment
          >> I'd be interested in peoples experiences with acceptance testing JSP.
          >
          >Your message suggests that you know about HttpUnit. Have you found a way
          to
          >unit test JSPs, though? I'm sure it's too late to debate architectural
          >choices (or, is it?), but I've found simple servlets and something like
          >WebMacro to be much easier to test at a lower level. Specifically, you can
          >have a WebMacro object (called a Page, perhaps) generate the output text
          >without having to deploy the application. Incredibly useful for
          >unit-testing Pages.

          I keep my JSP's extremely thin (they are effectively just the view). I
          unit test the model and controller extensively using mock requests and
          responses.

          I have used tools similar to WebMacro but the choice for this project is
          already made.


          >Try not to use abundant acceptance tests in place of abundant unit tests.

          I'm reasonably happy with our unit testing so far.

          >> - Do you use a dedicated database for acceptance testing?
          >
          >For me, tests that involve a database should run against an isolated
          >database. Nothing is more annoying than having two groups (pairs, teams,
          >whatever) running the same tests simultaneously against the same database.
          >We always ran our acceptance tests on the development box before
          >integrating, and we quickly decided to install Oracle on every box.

          That's probably not going to be an option for us for reasons I wont bother
          getting into.

          >> * How do you test things like email being sent?
          >
          >Your system should fail if JavaMail throws an exception at you, and that's
          >easy enough to detect. Beyond that, do you really want to assume
          >responsibility for the actual delivery of the mail? If so, perhaps build a
          >mock mailserver, and direct the mail to that. How to tie that in with a
          >passing test? That's left to the reader, yet again. I'm sure someone has
          >already written one.

          It's not so much that I'm assuming responsibility for it. Just want to
          ensure one gets sent. Redirecting mail to a file server will probably
          be enough.

          >> * What do you do about interactions external systems?
          >> - Stub them out somehow?
          >> - Set them up to a known state?
          >> - Pretent the problem doesn't exist?
          >
          >We had test copies of every external system. Make your acceptance and
          >integration environment look as much like the real system as possible.
          What
          >happens if you cannot replicate the same conditions? I don't know. XP
          >acceptance tests must be testable. Get the customer to describe how they
          >want to test the "...charge $2000.00 to *my* credit card..." story without
          >ruining anyone's credit. If they can't come up with something, perhaps the
          >team will need to drop back and unit test with mocks.

          The interfaces to our external systems are small and simple enough that it
          would probably be possible just to fake them.

          >> * How do you handle time based dependancies? Eg, user presses a button
          to
          >> schedule a test to run tomorrow.
          >
          >If the test depends upon the date, then I'd fake the date, rather than
          >delaying the test.

          Given that the system will probably be running in a seperate VM ie, the web
          server how would you fake the date?


          Thanks for your reply,

          Regards,

          Glen


          [Non-text portions of this message have been removed]
        • Glen Stampoultzis
          ... I tried it and found it somewhat difficult. Too much setup code. -- Glen Stampoultzis [Non-text portions of this message have been removed]
          Message 4 of 13 , Nov 29, 2001
          • 0 Attachment
            >Mocking up the DB is also popular, see for instance
            > http://www.mockobjects.com/papers/jdbc_testfirst.html

            I tried it and found it somewhat difficult. Too much setup code.

            -- Glen Stampoultzis


            [Non-text portions of this message have been removed]
          • Steve Freeman
            From: Glen Stampoultzis ... what s the alternative? Running against a copy of the database? What s the setup cost of that? Also, in any
            Message 5 of 13 , Nov 30, 2001
            • 0 Attachment
              From: "Glen Stampoultzis" <glens@...>
              > >Mocking up the DB is also popular, see for instance
              > > http://www.mockobjects.com/papers/jdbc_testfirst.html
              >
              > I tried it and found it somewhat difficult. Too much setup code.

              what's the alternative? Running against a copy of the database? What's the setup cost of that? Also, in any real test suite, one would factor the setup into some helper code, so the costs amortize across the tests pretty quickly.

              that said, we obviously have work to do to make the library easier to work with, but Sun keep making the API more complicated :-(

              Steve
            • Alain Ravet
              ... To me, the simplest thing is using an isolation layer; like a interface Persistor For unit testing, you just create class RamPersistor implements Persistor
              Message 6 of 13 , Nov 30, 2001
              • 0 Attachment
                --- "Steve Freeman" <steve@m...> wrote:
                > From: "Glen Stampoultzis" <glens@o...>
                > > >Mocking up the DB is also popular, see for instance
                > > > http://www.mockobjects.com/papers/jdbc_testfirst.html
                > >
                > > I tried it and found it somewhat difficult. Too much setup
                > > code.
                >
                > what's the alternative? Running against a copy of the database?


                To me, the simplest thing is using an isolation layer;
                like a
                interface Persistor


                For unit testing, you just create
                class RamPersistor implements Persistor

                that stores data in some java collections.

                Advantage :
                - refactoring friendly
                - unit tes friendly
                - super fast

                You'd still have to create/test a "real"
                class JdbcPersistor implements Persistor

                for the real thing, but this task can be delayed till later in the
                projet.



                Alain Ravet
              • Steve Freeman
                From: Alain Ravet ... Agreed, creating an intermediate object is almost always a Good Thing. If you re going to build one for
                Message 7 of 13 , Nov 30, 2001
                • 0 Attachment
                  From: "Alain Ravet" <aravet@...>
                  > --- "Steve Freeman" <steve@m...> wrote:
                  > > From: "Glen Stampoultzis" <glens@o...>
                  > > > >Mocking up the DB is also popular, see for instance
                  > > > > http://www.mockobjects.com/papers/jdbc_testfirst.html
                  > > >
                  > > > I tried it and found it somewhat difficult. Too much setup
                  > > > code.
                  > >
                  > > what's the alternative? Running against a copy of the database?
                  > To me, the simplest thing is using an isolation layer;
                  > like a
                  > interface Persistor
                  > For unit testing, you just create
                  > class RamPersistor implements Persistor
                  >
                  > that stores data in some java collections.
                  >
                  > Advantage :
                  > - refactoring friendly
                  > - unit tes friendly
                  > - super fast
                  >
                  > You'd still have to create/test a "real"
                  > class JdbcPersistor implements Persistor
                  >
                  > for the real thing, but this task can be delayed till later in the
                  > projet.

                  Agreed, creating an intermediate object is almost always a Good Thing. If you're going to build one for testing, then you might as well take a look at our expectation library because (we believe) that will make assertion testing easier with better error messages. Next question, how do you test your JdbcPersistor?

                  Steve
                • Alain Ravet
                  ... It s the best... in most cases. ... I m not sure. Initially, I just need persistence; I don t care if it s jdbc, xml, ODBMS, cvs, flat file... Our
                  Message 8 of 13 , Nov 30, 2001
                  • 0 Attachment
                    --- "Steve Freeman" <steve@m...> wrote:
                    > Agreed, creating an intermediate object is almost always a
                    > Good Thing.

                    It's the best... in most cases.


                    > If you're going to build one for testing, then you might as well
                    > take a look at our expectation library because (we believe) that
                    > will make assertion testing easier with better error messages.


                    I'm not sure. Initially, I just need persistence; I don't care if
                    it's jdbc, xml, ODBMS, cvs, flat file...

                    Our persistence layer looks like this :

                    interface Persistor
                    boolean persist (Order anOrder)..
                    boolean unpersist (Order anOrder)..
                    Order retrieveOrder (int id) ...

                    It doesn't say jdbc anywhere.
                    It hides the implementation so well that we haven't chosen yet : exml
                    (themindelectric.com), or jdbc.
                    The exml solution came to us because it's so simple : it persists
                    with "put", and retrieves with "get". Just like our RamPersistor.
                    If it's good enough for our current very simple project, we could go
                    that way.


                    > Next question, how do you test your JdbcPersistor?

                    That's the good one.
                    Another reason why we are looking into the exml solution.


                    Alain


                    >



                    --- In extremeprogramming@y..., "Steve Freeman" <steve@m...> wrote:
                    > From: "Alain Ravet" <aravet@c...>
                    > > --- "Steve Freeman" <steve@m...> wrote:
                    > > > From: "Glen Stampoultzis" <glens@o...>
                    > > > > >Mocking up the DB is also popular, see for instance
                    > > > > > http://www.mockobjects.com/papers/jdbc_testfirst.html
                    > > > >
                    > > > > I tried it and found it somewhat difficult. Too much
                    setup
                    > > > > code.
                    > > >
                    > > > what's the alternative? Running against a copy of the
                    database?
                    > > To me, the simplest thing is using an isolation layer;
                    > > like a
                    > > interface Persistor
                    > > For unit testing, you just create
                    > > class RamPersistor implements Persistor
                    > >
                    > > that stores data in some java collections.
                    > >
                    > > Advantage :
                    > > - refactoring friendly
                    > > - unit tes friendly
                    > > - super fast
                    > >
                    > > You'd still have to create/test a "real"
                    > > class JdbcPersistor implements Persistor
                    > >
                    > > for the real thing, but this task can be delayed till later in
                    the
                    > > projet.
                    >
                    > Agreed, creating an intermediate object is almost always a Good
                    Thing. If you're going to build one for testing, then you might as
                    well take a look at our expectation library because (we believe) that
                    will make assertion testing easier with better error messages. Next
                    question, how do you test your JdbcPersistor?
                    >
                    > Steve
                  • Alain Ravet
                    ... It s the best... in most cases. ... I m not sure. Initially, I just need persistence; I don t care if it s jdbc, xml, ODBMS, cvs, flat file... Our
                    Message 9 of 13 , Nov 30, 2001
                    • 0 Attachment
                      --- "Steve Freeman" <steve@m...> wrote:
                      > Agreed, creating an intermediate object is almost always a
                      > Good Thing.

                      It's the best... in most cases.


                      > If you're going to build one for testing, then you might as well
                      > take a look at our expectation library because (we believe) that
                      > will make assertion testing easier with better error messages.


                      I'm not sure. Initially, I just need persistence; I don't care if
                      it's jdbc, xml, ODBMS, cvs, flat file...

                      Our persistence layer looks like this :

                      interface Persistor
                      boolean persist (Order anOrder)..
                      boolean unpersist (Order anOrder)..
                      Order retrieveOrder (int id) ...

                      It doesn't say jdbc anywhere.
                      It hides the implementation so well that we haven't chosen yet : exml
                      (themindelectric.com), or jdbc.
                      The exml solution came to us because it's so simple : it persists
                      with "put", and retrieves with "get". Just like our RamPersistor.
                      If it's good enough for our current very simple project, we could go
                      that way.


                      > Next question, how do you test your JdbcPersistor?

                      That's the good one.
                      Another reason why we are looking into the exml solution.


                      Alain


                      >



                      --- In extremeprogramming@y..., "Steve Freeman" <steve@m...> wrote:
                      > From: "Alain Ravet" <aravet@c...>
                      > > --- "Steve Freeman" <steve@m...> wrote:
                      > > > From: "Glen Stampoultzis" <glens@o...>
                      > > > > >Mocking up the DB is also popular, see for instance
                      > > > > > http://www.mockobjects.com/papers/jdbc_testfirst.html
                      > > > >
                      > > > > I tried it and found it somewhat difficult. Too much
                      setup
                      > > > > code.
                      > > >
                      > > > what's the alternative? Running against a copy of the
                      database?
                      > > To me, the simplest thing is using an isolation layer;
                      > > like a
                      > > interface Persistor
                      > > For unit testing, you just create
                      > > class RamPersistor implements Persistor
                      > >
                      > > that stores data in some java collections.
                      > >
                      > > Advantage :
                      > > - refactoring friendly
                      > > - unit tes friendly
                      > > - super fast
                      > >
                      > > You'd still have to create/test a "real"
                      > > class JdbcPersistor implements Persistor
                      > >
                      > > for the real thing, but this task can be delayed till later in
                      the
                      > > projet.
                      >
                      > Agreed, creating an intermediate object is almost always a Good
                      Thing. If you're going to build one for testing, then you might as
                      well take a look at our expectation library because (we believe) that
                      will make assertion testing easier with better error messages. Next
                      question, how do you test your JdbcPersistor?
                      >
                      > Steve
                    • Rob Myers
                      Glen, ... Ah, you have an acceptance test that spans days. Can you give an example? Solutions are easier when we understand the problem. ( Great cop-out,
                      Message 10 of 13 , Nov 30, 2001
                      • 0 Attachment
                        Glen,

                        > Given that the system will probably be running in a seperate VM
                        > ie, the web
                        > server how would you fake the date?

                        Ah, you have an acceptance test that spans days. Can you give an example?
                        Solutions are easier when we understand the problem.

                        ("Great cop-out, Rob."
                        "Thanks! I thought so...")

                        ;)

                        Rob

                        Rob Myers
                        XP Coach/Programmer
                        Mindful Software
                        www.mindfulsoftware.com
                      • Steve Freeman
                        From: Alain Ravet ... I m very happy for you ;-) ... well, if you do need to test some JDBC, perhaps we can talk again... Steve
                        Message 11 of 13 , Nov 30, 2001
                        • 0 Attachment
                          From: "Alain Ravet" <aravet@...>
                          > Our persistence layer looks like this :
                          >
                          > interface Persistor
                          > boolean persist (Order anOrder)..
                          > boolean unpersist (Order anOrder)..
                          > Order retrieveOrder (int id) ...
                          >
                          > It doesn't say jdbc anywhere.
                          > It hides the implementation so well that we haven't chosen yet : exml
                          > (themindelectric.com), or jdbc.
                          > The exml solution came to us because it's so simple : it persists
                          > with "put", and retrieves with "get". Just like our RamPersistor.
                          > If it's good enough for our current very simple project, we could go
                          > that way.

                          I'm very happy for you ;-)

                          > > Next question, how do you test your JdbcPersistor?
                          > That's the good one.
                          > Another reason why we are looking into the exml solution.

                          well, if you do need to test some JDBC, perhaps we can talk again...

                          Steve
                        • Steve Hayes
                          ... I wrote a servlet that lets me change the effective date of the application. It only gets invoked from the acceptance tests, but it wasn t a big deal.
                          Message 12 of 13 , Nov 30, 2001
                          • 0 Attachment
                            I've missed a post somewhere - I think this was from Glen:
                            >
                            > > Given that the system will probably be running in a seperate VM
                            > > ie, the web
                            > > server how would you fake the date?

                            I wrote a servlet that lets me change the effective date of the
                            application. It only gets invoked from the acceptance tests, but it wasn't
                            a big deal.

                            Steve Hayes
                          • john.link@gmx.net
                            ... And less testable. It s amazing how untestable an API can be (JDBC) *although* it s composed almost exclusively of interfaces. Johannes
                            Message 13 of 13 , Dec 2, 2001
                            • 0 Attachment
                              > that said, we obviously have work to do to make the library easier
                              > to work with, but Sun keep making the API more complicated :-(

                              And less testable. It's amazing how untestable an API can be (JDBC)
                              *although* it's composed almost exclusively of interfaces.

                              Johannes
                            Your message has been successfully submitted and would be delivered to recipients shortly.