Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] IS agenda testing patterns

Expand Messages
  • Curtis Cooley
    ... patterns, so even if you decide to roll your own framework, you ll get some good info. At a high level, you want a way to specify test data, FitNesse uses
    Message 1 of 8 , Jan 19, 2010
    • 0 Attachment
      On Tue, Jan 19, 2010 at 5:44 AM, strazhce <infobox.oleg@...> wrote:

      >
      >
      > Hi.
      >
      > I'm trying to figure out the proper structure of tests for our project. I
      > have several ideas and I wonder, if you know some tested patterns for doing
      > that.
      >
      > My situation:
      > I have a web application, which manages contracts. There are few type of
      > agendas - create contract, change contract, cancel contract etc. Every
      > agenda has a wizard-like sequence of screens, each one may have some
      > subscreens tied to button or some actions based on buttons (like "add item"
      > etc.).
      > We have tests, which are pretty monolithic - e.g.
      > CreateContractForProductX, which has most parameters as fields with defaults
      > plus some conditionals (e.g. if flag X is set, do something different). This
      > is working ok, but is no longer sufficient, not flexible enough etc.
      >
      > What I need:
      > Tests, which are able to create/modify contracts in GUI. Contracts may be
      > complex, so I need to specify e.g. some base contract with predefined
      > parameters and have ability to e.g. specify different set of products,
      > discounts etc.
      > The purpose of these tests is functional/acceptance/integration/regression
      > testing.
      >
      > Approach:
      > I see two possible approaches:
      > 1. Specify target state, let test do the work.
      > 2. Specify operations, let dumb test to execute them.
      >
      > Questions:
      > 1. Did you solve similar problem?
      > 2. What was the essence of your approach?
      > 3. What patterns emerged from your solution?
      > 4. Any other ideas/observations?
      >
      > Thanks.
      > Oleg
      >
      > I'd suggest taking a look at FitNesse or Cucumber. At http://fitnesse.orgthere is a ton of written documentation on testing and acceptance test
      patterns, so even if you decide to roll your own framework, you'll get some
      good info.

      At a high level, you want a way to specify test data, FitNesse uses html
      tables. Then you want a way to feed that data into your test fixtures. Then
      you need a way to compare the output to the expected output. Again FitNesse
      uses the same html tables it uses for input. Also, find a way to exercise
      your application without going through the GUI. If you are willing to write
      or rewrite a bunch of acceptance tests, also take the time to seperate your
      GUI layer from your application layer so you can test easier.

      --
      Curtis Cooley
      curtis.cooley@...
      home:http://curtiscooley.com
      blog:http://ponderingobjectorienteddesign.blogspot.com
      ===============
      Leadership is a potent combination of strategy and character. But if you
      must be without one, be without the strategy.
      -- H. Norman Schwarzkopf


      [Non-text portions of this message have been removed]
    • strazhce
      Hi, Andre. Very good questions, I m considering this as well. ... Why - I don t know how to do that through controller only properly. I think we could pull
      Message 2 of 8 , Jan 19, 2010
      • 0 Attachment
        Hi, Andre.

        Very good questions, I'm considering this as well.

        --- In extremeprogramming@yahoogroups.com, D.André Dhondt <d.andre.dhondt@...> wrote:
        >
        > Why do you need tests that hit the GUI directly? Is your business logic in
        > a testable layer? Can you imagine how to separate the functional tests from
        > the integration tests?

        Why - I don't know how to do that through controller only properly.

        I think we could pull strings of our M/C of MVC, but it would be very hard :-(
        - controller API is very advanced, with strict separation of View from MC, but also complex at the same time
        - certain validations/controller API calls are tied to GUI transitions and events generated from there - we would have to fire those events manually in tests
        - our current skills and application related utilities for unit testing are not very good. We would have to change testing paradigm of the team from GUI oriented testing to something else, that's hard. That's a large topic for itself.

        Is it common to create another layer around controller for unit/integration testing purposes, to simplify things? Something similar to what I want for GUI testing? I don't see another way around those complexities above. But that's why I post here ;-)

        Oleg
      • Curtis Cooley
        ... lost count of the times and number of agilists I ve heard advise making the test the first interface and the GUI the second. That does not help you now,
        Message 3 of 8 , Jan 19, 2010
        • 0 Attachment
          On Tue, Jan 19, 2010 at 3:07 PM, strazhce <infobox.oleg@...> wrote:

          > Is it common to create another layer around controller for
          > unit/integration testing purposes, to simplify things? Something similar to
          > what I want for GUI testing? I don't see another way around those
          > complexities above. But that's why I post here ;-)
          >
          > Not only common but the recommended way to do this kind of testing. I've
          lost count of the times and number of agilists I've heard advise making the
          test the first interface and the GUI the second.

          That does not help you now, sadly, since your GUI is already built, but
          incrementally attacking that complex controller interface to make it more
          testable might also lead to some other benefits as well.

          Automated Monkey Testing (Robotest for example) is so fragile and unruly
          that I rarely see it mentioned as an option, let alone put into production
          enough times by the same person/people to have reliable and useful patterns
          emerge. Where as FitNesse for example has been used over and over by the
          same groups of people that good patterns are starting to emerge.

          --
          Curtis Cooley
          curtis.cooley@...
          home:http://curtiscooley.com
          blog:http://ponderingobjectorienteddesign.blogspot.com
          ===============
          Leadership is a potent combination of strategy and character. But if you
          must be without one, be without the strategy.
          -- H. Norman Schwarzkopf


          [Non-text portions of this message have been removed]
        • George Dinwiddie
          Oleg, I suggest extracting the business logic into a separate tier. It then becomes the model for the MVC pattern. See http://bit.ly/6K2Nhp for more
          Message 4 of 8 , Jan 19, 2010
          • 0 Attachment
            Oleg,

            I suggest extracting the business logic into a separate tier. It then
            becomes the model for the MVC pattern. See http://bit.ly/6K2Nhp for
            more discussion on this.

            This lets you test the business rules directly against this layer. Only
            the navigation through the app needs to be tested through the GUI.

            - George

            strazhce wrote:
            > Hi, Andre.
            >
            > Very good questions, I'm considering this as well.
            >
            > --- In extremeprogramming@yahoogroups.com, D.André Dhondt <d.andre.dhondt@...> wrote:
            >> Why do you need tests that hit the GUI directly? Is your business logic in
            >> a testable layer? Can you imagine how to separate the functional tests from
            >> the integration tests?
            >
            > Why - I don't know how to do that through controller only properly.
            >
            > I think we could pull strings of our M/C of MVC, but it would be very hard :-(
            > - controller API is very advanced, with strict separation of View
            > from MC, but also complex at the same time
            > - certain validations/controller API calls are tied to GUI
            > transitions and events generated from there - we would have to fire
            > those events manually in tests
            > - our current skills and application related utilities for unit
            > testing are not very good. We would have to change testing paradigm
            > of the team from GUI oriented testing to something else, that's hard.
            > That's a large topic for itself.
            >
            > Is it common to create another layer around controller for
            > unit/integration testing purposes, to simplify things? Something
            > similar to what I want for GUI testing? I don't see another way
            > around those complexities above. But that's why I post here ;-)


            --
            ----------------------------------------------------------------------
            * George Dinwiddie * http://blog.gdinwiddie.com
            Software Development http://www.idiacomputing.com
            Consultant and Coach http://www.agilemaryland.org
            ----------------------------------------------------------------------
          • D.André Dhondt
            ... Anything of value has its price ;) Take baby steps, hit just a flaky test, or just one slow/expensive test, or even a test that could be written easily to
            Message 5 of 8 , Jan 19, 2010
            • 0 Attachment
              On Wed, Jan 20, 2010 at 12:07 AM, strazhce <infobox.oleg@...> wrote:

              > I think we could pull strings of our M/C of MVC, but it would be very hard
              > :-(
              >
              Anything of value has its price ;)
              Take baby steps, hit just a flaky test, or just one slow/expensive test, or
              even a test that could be written easily to find out if decoupling it from
              the gui helps. Do something that is important for the current iteration,
              and slowly the team can re-align the tests to the new vision of testing, or
              improve that vision, then re-align...


              > - certain validations/controller API calls are tied to GUI transitions and
              > events generated from there - we would have to fire those events manually in
              > tests
              >
              I don't have a problem with firing events directly from test code. There is
              some risk that the event isn't wired up to the GUI, but regressions there
              are rare (in my experience) so I don't bother automating gui event tests.
              Instead, I test the behavior of the event itself, since that's what I care
              about.


              > - our current skills and application related utilities for unit testing are
              > not very good. We would have to change testing paradigm of the team from GUI
              > oriented testing to something else, that's hard. That's a large topic for
              > itself.
              >
              It is, but instead of changing the whole team, reduce scope... just try one
              test, show it to people, ask for their help, discuss, weigh the pros and
              cons, and make it better... or attack the next test, together. Bit by bit,
              the team's vision of the testing paradigm will change all by itself.



              --
              D. André Dhondt
              http://dhondtsayitsagile.blogspot.com/

              Support low-cost conferences -- http://agiletour.org/
              If you're in the area, join Agile Philly http://www.AgilePhilly.com
              Mentor/be mentored: the Agile Skills Project
              https://sites.google.com/site/agileskillsprojectwiki/


              [Non-text portions of this message have been removed]
            • JeffGrigg
              In my experience of retrofitting tests into legacy code, I find it helpful to think of the system having a stack of layers like this: 0. The user 1. Views 2.
              Message 6 of 8 , Jan 20, 2010
              • 0 Attachment
                In my experience of retrofitting tests into legacy code, I find it helpful to think of the system having a stack of layers like this:

                0. The user
                1. Views
                2. Presenter/Controller
                3. Business Logic
                4. Data Access Objects
                5. Database / data store

                There are useful tools, like Selenium, that will test the whole stack. But they're slow and brittle, as the system changes.

                So I end up writing mocks/fakes for the top and bottom layers (1. Views and 5. Database) and testing everything in between (2 thru 4).

                Use weak Singletons (instances replaceable by test code) and possibly factory objects stored as Singletons to mock the lowest levels -- particularly when they're called through static functions. It's unpleasant, but it gets the tests working. And you can refactor the Singleton-ness out later, as needed.


                --- "strazhce" <infobox.oleg@...> wrote:
                > - controller API is very advanced, with strict separation of
                > View from MC, but also complex at the same time

                This sounds like good news: If well separated, then you should be able to mock the view layer easily.

                > - certain validations/controller API calls are tied to GUI
                > transitions and events generated from there - we would
                > have to fire those events manually in tests

                Yes, you'll have to simulate GUI events from your tests.

                > - our current skills and application related utilities for
                > unit testing are not very good. We would have to change
                > testing paradigm of the team from GUI oriented testing
                > to something else, that's hard. [...]

                Look at it as on opportunity to learn. And don't think you're limited to just one paradigm: You can have unit tests, integration tests and acceptance tests all working together to test a single system! I'll often have mock-based integration tests that display no GUI and do not hit the database, and also run Selenium tests against the full stack, with GUI and test database. You can do more than one thing. And during transitions, you'll have to.

                > Is it common to create another layer around controller for
                > unit/integration testing purposes, to simplify things?

                I say mock the view. Use the real controller and business logic. Mock the view and the database.

                Thank you for the questions! We're all glad to help!
                - jeff
              Your message has been successfully submitted and would be delivered to recipients shortly.