Loading ...
Sorry, an error occurred while loading the content.
 

Re: [agile-usability] How do you unit test web interfaces?

Expand Messages
  • Phlip
    ... Convince your HTML generating modules to generate HTML as if a web browser had called them. Then use the XPath query technique to reach into the HTML and
    Message 1 of 22 , Apr 24, 2007
      > > None of these are for _unit_ tests. Please write as many
      > > tests as possible to the HTML layer, or lower, for best
      > > results. Don't be fooled by pretty special effects.
      >
      > Phlip, can you explain more precisely what you mean by "writing a test
      > to the HTML layer or lower". I think I know what you mean, but want to
      > be sure.

      Convince your HTML generating modules to generate HTML as if a web
      browser had called them. Then use the XPath query technique to reach
      into the HTML and look for details. This probably covers that:

      http://idiacomputing.com/moin/HtmlTestingUsingXpath

      --
      Phlip
      http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!
    • Phlip
      ... server ... ! -- Phlip http://c2.com/cgi/wiki?ZeekLand
      Message 2 of 22 , Apr 24, 2007
        > Convince your HTML generating modules to generate HTML as if a web

        server

        > had called them. Then use the XPath query technique to reach
        > into the HTML and look for details. This probably covers that:
        >
        > http://idiacomputing.com/moin/HtmlTestingUsingXpath

        !

        --
        Phlip
        http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!
      • Brian Marick
        ... I have something of an example here, using the Atomic Object s variant of whatever-Martin-Fowler-calls-Model/View/Presenter-these-days:
        Message 3 of 22 , Apr 24, 2007
          On Apr 24, 2007, at 8:29 AM, Desilets, Alain wrote:

          > Phlip, can you explain more precisely what you mean by "writing a test
          > to the HTML layer or lower". I think I know what you mean, but want to
          > be sure.

          I have something of an example here, using the Atomic Object's
          variant of whatever-Martin-Fowler-calls-Model/View/Presenter-these-days:

          <http://www.testing.com/cgi-bin/blog/2007/01/05#wireframe2>
          <http://www.testing.com/cgi-bin/blog/2007/01/11#wireframe3>
          <http://www.testing.com/cgi-bin/blog/2007/01/18#wireframe4>

          This example assumes a non-HTML app, but I plan to repurpose it to
          HTML to show that's tractable. (Haven't started that yet.) The idea
          of MVP (note: different than MVC) is to make the view very thin and
          put all the view smarts ("when text is entered in this text box, make
          that button visible") into a layer below it. The first link is to a
          movie that explains the Atomic Object style, which uses the Observer
          pattern heavily.

          See also Mike Feather's "Humble Dialog Box":
          <http://www.objectmentor.com/resources/articles/TheHumbleDialogBox.pdf>

          The Atomic Object style uses mocks heavily. I was a little surprised
          to find myself deviating from that. What happened is that I found
          that business-facing tests written as annotated wireframes (and used
          Fit as an execution engine) seemed like just as good a way to test-
          drive the presenter layer, and less work besides.

          I need to get back to that project. So many opportunities to
          experiment, so little time...

          -----
          Brian Marick, independent consultant
          Mostly on agile methods with a testing slant
          www.exampler.com, www.exampler.com/blog
        • Desilets, Alain
          OK, thx. That was the next step up I was thinking of taking, i.e. instead of just diffing the HTML against a known past output, just assert that the HTML
          Message 4 of 22 , Apr 24, 2007
            OK, thx. That was the next step up I was thinking of taking, i.e.
            instead of just diffing the HTML against a known past output, just
            assert that the HTML contains specific things I am expecting to find.

            This would certainly address my first problem (i.e. all my tests failing
            as soon as the HTML container is changed).

            But what about the other issue I was raising, i.e. the fact that you are
            not testing pathes through various HTML pages.

            In the past, I have been burned badly by this. For example, I would have
            one HTML page whose URL was:

            http://www.somewhere.com/cgi-bin/a-script.cgi

            that was supposed to have a link to a page with URL:

            http://www.somewhere.com/cgi-bin/another-script.cgi?some-argument=1

            I would test that both the a-script.cgi and
            another-script.cgi?some-argument=1 pages produced the expected HTML.

            Then, I would refactor the name of the argument to another-script, so
            that its URL would be:

            http://www.somewhere.com/cgi-bin/another-script.cgi?refactored-argument-
            name=1

            My tests would all run fine, but when I would test interactively
            (something I found myself doing far too often), I would realize that the
            first HTML now had a dead link that pointed nowhere known.

            I guess with the Xpath approach, this is less likely to happen, because
            I can define some variable which all tests can access, which says
            something like this:

            my $argument_name = 'some-argument';

            And use that in my assertions.

            Are there other common pitfalls and ways to avoid them?

            Alain

            > -----Original Message-----
            > From: agile-usability@yahoogroups.com
            > [mailto:agile-usability@yahoogroups.com] On Behalf Of Phlip
            > Sent: April 24, 2007 11:17 AM
            > To: agile-usability@yahoogroups.com
            > Subject: Re: [agile-usability] How do you unit test web interfaces?
            >
            > > > None of these are for _unit_ tests. Please write as many
            > > tests
            > > as possible to the HTML layer, or lower, for best >
            > results. Don't be
            > > fooled by pretty special effects.
            > >
            > > Phlip, can you explain more precisely what you mean by "writing a
            > > test to the HTML layer or lower". I think I know what you
            > mean, but
            > > want to be sure.
            >
            > Convince your HTML generating modules to generate HTML as if
            > a web browser had called them. Then use the XPath query
            > technique to reach into the HTML and look for details. This
            > probably covers that:
            >
            > http://idiacomputing.com/moin/HtmlTestingUsingXpath
            >
            > --
            > Phlip
            > http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!
            >
            >
            >
            > Yahoo! Groups Links
            >
            >
            >
            >
          • George Dinwiddie
            Alain, What is the focus of your testing? Are you trying to determine if a given query produces the correct output? Or are you trying to test a Use Case
            Message 5 of 22 , Apr 24, 2007
              Alain,

              What is the focus of your testing? Are you trying to determine if a
              given query produces the correct output? Or are you trying to test a
              Use Case through a series of interactions with the system?

              - George

              Desilets, Alain wrote:
              > OK, thx. That was the next step up I was thinking of taking, i.e.
              > instead of just diffing the HTML against a known past output, just
              > assert that the HTML contains specific things I am expecting to find.
              >
              > This would certainly address my first problem (i.e. all my tests failing
              > as soon as the HTML container is changed).
              >
              > But what about the other issue I was raising, i.e. the fact that you are
              > not testing pathes through various HTML pages.
              >
              > In the past, I have been burned badly by this. For example, I would have
              > one HTML page whose URL was:
              >
              > http://www.somewhere.com/cgi-bin/a-script.cgi
              >
              > that was supposed to have a link to a page with URL:
              >
              > http://www.somewhere.com/cgi-bin/another-script.cgi?some-argument=1
              >
              > I would test that both the a-script.cgi and
              > another-script.cgi?some-argument=1 pages produced the expected HTML.
              >
              > Then, I would refactor the name of the argument to another-script, so
              > that its URL would be:
              >
              > http://www.somewhere.com/cgi-bin/another-script.cgi?refactored-argument-
              > name=1
              >
              > My tests would all run fine, but when I would test interactively
              > (something I found myself doing far too often), I would realize that the
              > first HTML now had a dead link that pointed nowhere known.
              >
              > I guess with the Xpath approach, this is less likely to happen, because
              > I can define some variable which all tests can access, which says
              > something like this:
              >
              > my $argument_name = 'some-argument';
              >
              > And use that in my assertions.
              >
              > Are there other common pitfalls and ways to avoid them?
              >
              > Alain
              >
              >> -----Original Message-----
              >> From: agile-usability@yahoogroups.com
              >> [mailto:agile-usability@yahoogroups.com] On Behalf Of Phlip
              >> Sent: April 24, 2007 11:17 AM
              >> To: agile-usability@yahoogroups.com
              >> Subject: Re: [agile-usability] How do you unit test web interfaces?
              >>
              >>> > None of these are for _unit_ tests. Please write as many
              >> > tests
              >>> as possible to the HTML layer, or lower, for best >
              >> results. Don't be
              >>> fooled by pretty special effects.
              >>>
              >>> Phlip, can you explain more precisely what you mean by "writing a
              >>> test to the HTML layer or lower". I think I know what you
              >> mean, but
              >>> want to be sure.
              >> Convince your HTML generating modules to generate HTML as if
              >> a web browser had called them. Then use the XPath query
              >> technique to reach into the HTML and look for details. This
              >> probably covers that:
              >>
              >> http://idiacomputing.com/moin/HtmlTestingUsingXpath
              >>
              >> --
              >> Phlip
              >> http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!
              >>
              >>
              >>
              >> Yahoo! Groups Links
              >>
              >>
              >>
              >>
              >
              >
              >
              > Yahoo! Groups Links
              >
              >
              >
              >


              --
              ----------------------------------------------------------------------
              * George Dinwiddie * http://blog.gdinwiddie.com
              Software Development http://www.idiacomputing.com
              Consultant and Coach http://www.agilemaryland.org
              ----------------------------------------------------------------------
            • Desilets, Alain
              ... Both. In some case, I may even need to test that the javascript embedded in the HTML is working properly. Alain
              Message 6 of 22 , Apr 24, 2007
                > Alain,
                >
                > What is the focus of your testing? Are you trying to
                > determine if a given query produces the correct output? Or
                > are you trying to test a Use Case through a series of
                > interactions with the system?

                Both.

                In some case, I may even need to test that the javascript embedded in
                the HTML is working properly.

                Alain
              • George Dinwiddie
                ... This all sounds rather far from the concept of unit testing as defined in Test Driven Development. It sounds more like acceptance or customer testing, to
                Message 7 of 22 , Apr 24, 2007
                  Desilets, Alain wrote:
                  >> Alain,
                  >>
                  >> What is the focus of your testing? Are you trying to
                  >> determine if a given query produces the correct output? Or
                  >> are you trying to test a Use Case through a series of
                  >> interactions with the system?
                  >
                  > Both.
                  >
                  > In some case, I may even need to test that the javascript embedded in
                  > the HTML is working properly.

                  This all sounds rather far from the concept of unit testing as defined
                  in Test Driven Development. It sounds more like acceptance or customer
                  testing, to me. For that, one of the tools to run a real browser (like
                  WATIR) is probably the way to go. I'd ask on the agile-testing list. I
                  think that you'll get better results, though, if you think through what
                  you're trying to accomplish. "Testing everything" at the GUI is not
                  often a very productive strategy.

                  If you're trying to TDD the gui, then you'll find that a variety of
                  tools are needed, as there is no SwissArmyUnit(tm) tool that does
                  everything. For TDDing javascript, I've found jsUnit useful.

                  - George

                  --
                  ----------------------------------------------------------------------
                  * George Dinwiddie * http://blog.gdinwiddie.com
                  Software Development http://www.idiacomputing.com
                  Consultant and Coach http://www.agilemaryland.org
                  ----------------------------------------------------------------------
                • Desilets, Alain
                  ... I agree that testing pathes through several pages is acceptance testing, not unit testing. But I think testing the behaviour of a single web page
                  Message 8 of 22 , Apr 25, 2007
                    > > In some case, I may even need to test that the javascript
                    > embedded in
                    > > the HTML is working properly.
                    >
                    > This all sounds rather far from the concept of unit testing
                    > as defined in Test Driven Development.
                    > It sounds more like
                    > acceptance or customer testing, to me.


                    I agree that testing pathes through several pages is acceptance testing,
                    not unit testing.

                    But I think testing the behaviour of a single web page (including the
                    behaviour of it's embedded JavaScript code) fits the definition of unit
                    testing.

                    In any case, I have never worried too much about whether I am doing unit
                    or acceptance testing. I always do both, usually at the same time, and
                    often in the same TestCase files (the later being practice I am trying
                    to move away from).

                    > For that, one of the
                    > tools to run a real browser (like
                    > WATIR) is probably the way to go. I'd ask on the
                    > agile-testing list. I think that you'll get better results,
                    > though, if you think through what you're trying to
                    > accomplish. "Testing everything" at the GUI is not often a
                    > very productive strategy.

                    Thanks for the agile testing hint. I'll post something there (I didn't
                    know that list existed).

                    I'm not thinking about testing everything at the GUI level. I always
                    have more internal tests that check the internals independantly of the
                    GUI.

                    But the Gui is code too, and it therefore needs to be tested. I'm
                    looking for a better way to test it than I have been doing in the past.

                    > If you're trying to TDD the gui, then you'll find that a
                    > variety of tools are needed, as there is no SwissArmyUnit(tm)
                    > tool that does everything. For TDDing javascript, I've found
                    > jsUnit useful.

                    Yes, I know. Some collegues and I are currently developing a library of
                    "manipulators" for wxPython, to make it easier to programmatically
                    manipulate widgets in a wxPython application for the purpose of unit
                    testing. I'm surprised we had to write that ourselves.


                    Alain
                  • George Dinwiddie
                    ... It fits *some* definitions of unit testing, but not the one I generally use. I generally use the TDD concept of unit testing, rather than the ones
                    Message 9 of 22 , Apr 25, 2007
                      Desilets, Alain wrote:
                      > But I think testing the behaviour of a single web page (including the
                      > behaviour of it's embedded JavaScript code) fits the definition of unit
                      > testing.

                      It fits *some* definitions of unit testing, but not the one I generally
                      use. I generally use the TDD concept of unit testing, rather than the
                      ones generally accepted by the testing community. I don't call anything
                      "unit testing" if it requires deployment to run.

                      The overloading of the term "unit testing" has lead me to prefer
                      "programmer testing" instead.

                      > Thanks for the agile testing hint. I'll post something there (I didn't
                      > know that list existed).
                      >
                      > I'm not thinking about testing everything at the GUI level. I always
                      > have more internal tests that check the internals independantly of the
                      > GUI.
                      >
                      > But the Gui is code too, and it therefore needs to be tested. I'm
                      > looking for a better way to test it than I have been doing in the past.

                      Yes, and you'll find a good community of testing professionals over
                      there who will give you good testing advice. There's also a few of us
                      developers there, who tend to look at things from the "TDD as a design
                      tool" point of view.

                      - George

                      --
                      ----------------------------------------------------------------------
                      * George Dinwiddie * http://blog.gdinwiddie.com
                      Software Development http://www.idiacomputing.com
                      Consultant and Coach http://www.agilemaryland.org
                      ----------------------------------------------------------------------
                    • Desilets, Alain
                      ... I don t really want to argue about the definition of unit vs acceptance testing. I agree with you that testing which requires deployment sucks, especially
                      Message 10 of 22 , Apr 25, 2007
                        > Desilets, Alain wrote:
                        > > But I think testing the behaviour of a single web page
                        > (including the
                        > > behaviour of it's embedded JavaScript code) fits the definition of
                        > > unit testing.
                        >
                        > It fits *some* definitions of unit testing, but not the one I
                        > generally use. I generally use the TDD concept of unit
                        > testing, rather than the ones generally accepted by the
                        > testing community. I don't call anything "unit testing" if
                        > it requires deployment to run.

                        I don't really want to argue about the definition of unit vs acceptance testing.

                        I agree with you that testing which requires deployment sucks, especially if it's manual deployment. I'm looking for tools and techniques that will allow me to do automated testing (whether it be unit or acceptance) of the GUI without requiring too much in the way of deployment. It seems to me that testing the embedded javascript should be possible without full deployment. For example, invoke the script that generates the HTML containing that javascript, then load that HTML into some kind of browser emulator that has a JavaScript interpretor, etc... Dunno if that exists or not.


                        ----
                        Alain Désilets, MASc
                        Agent de recherches/Research Officer
                        Institut de technologie de l'information du CNRC /
                        NRC Institute for Information Technology

                        alain.desilets@...
                        Tél/Tel (613) 990-2813
                        Facsimile/télécopieur: (613) 952-7151

                        Conseil national de recherches Canada, M50, 1200 chemin Montréal,
                        Ottawa (Ontario) K1A 0R6
                        National Research Council Canada, M50, 1200 Montreal Rd., Ottawa, ON
                        K1A 0R6

                        Gouvernement du Canada | Government of Canada
                      • Desilets, Alain
                        ... Thx Brian. I ll have a look. ... Right. When I test non-web GUIs, that s the approach I use. Except that contrarily to Mike, I don t use a MockView. I find
                        Message 11 of 22 , Apr 25, 2007
                          > I have something of an example here, using the Atomic
                          > Object's variant of
                          > whatever-Martin-Fowler-calls-Model/View/Presenter-these-days:
                          >
                          > <http://www.testing.com/cgi-bin/blog/2007/01/05#wireframe2>
                          > <http://www.testing.com/cgi-bin/blog/2007/01/11#wireframe3>
                          > <http://www.testing.com/cgi-bin/blog/2007/01/18#wireframe4>
                          >
                          > This example assumes a non-HTML app, but I plan to repurpose
                          > it to HTML to show that's tractable. (Haven't started that
                          > yet.) The idea of MVP (note: different than MVC) is to make
                          > the view very thin and put all the view smarts ("when text is
                          > entered in this text box, make that button visible") into a
                          > layer below it. The first link is to a movie that explains
                          > the Atomic Object style, which uses the Observer pattern heavily.

                          Thx Brian. I'll have a look.

                          >
                          > See also Mike Feather's "Humble Dialog Box":
                          > <http://www.objectmentor.com/resources/articles/TheHumbleDialo
                          > gBox.pdf>

                          Right. When I test non-web GUIs, that's the approach I use. Except that contrarily to Mike, I don't use a MockView. I find it's just as easy to test using the actual graphical view. And doing it that way catches all sort of silly little bugs in the view. Things like a button not being properly wired to the model method it's supposed to invoke.

                          I haven't figured out how to do this in a web context though, hence my post.


                          ----
                          Alain Désilets, MASc
                          Agent de recherches/Research Officer
                          Institut de technologie de l'information du CNRC /
                          NRC Institute for Information Technology

                          alain.desilets@...
                          Tél/Tel (613) 990-2813
                          Facsimile/télécopieur: (613) 952-7151

                          Conseil national de recherches Canada, M50, 1200 chemin Montréal,
                          Ottawa (Ontario) K1A 0R6
                          National Research Council Canada, M50, 1200 Montreal Rd., Ottawa, ON
                          K1A 0R6

                          Gouvernement du Canada | Government of Canada
                        • George Dinwiddie
                          ... If you want to test just the javascript, put it in a .js file and test it with jsUnit. -- ... * George Dinwiddie *
                          Message 12 of 22 , Apr 25, 2007
                            Desilets, Alain wrote:
                            > I agree with you that testing which requires deployment sucks,
                            > especially if it's manual deployment. I'm looking for tools and
                            > techniques that will allow me to do automated testing (whether it be
                            > unit or acceptance) of the GUI without requiring too much in the way
                            > of deployment. It seems to me that testing the embedded javascript
                            > should be possible without full deployment. For example, invoke the
                            > script that generates the HTML containing that javascript, then load
                            > that HTML into some kind of browser emulator that has a JavaScript
                            > interpretor, etc... Dunno if that exists or not.

                            If you want to test just the javascript, put it in a .js file and test
                            it with jsUnit.

                            --
                            ----------------------------------------------------------------------
                            * George Dinwiddie * http://blog.gdinwiddie.com
                            Software Development http://www.idiacomputing.com
                            Consultant and Coach http://www.agilemaryland.org
                            ----------------------------------------------------------------------
                          Your message has been successfully submitted and would be delivered to recipients shortly.