Loading ...
Sorry, an error occurred while loading the content.

IS agenda testing patterns

Expand Messages
  • strazhce
    Hi. I m trying to figure out the proper structure of tests for our project. I have several ideas and I wonder, if you know some tested patterns for doing that.
    Message 1 of 8 , Jan 19, 2010
    • 0 Attachment
      Hi.

      I'm trying to figure out the proper structure of tests for our project. I have several ideas and I wonder, if you know some tested patterns for doing that.

      My situation:
      I have a web application, which manages contracts. There are few type of agendas - create contract, change contract, cancel contract etc. Every agenda has a wizard-like sequence of screens, each one may have some subscreens tied to button or some actions based on buttons (like "add item" etc.).
      We have tests, which are pretty monolithic - e.g. CreateContractForProductX, which has most parameters as fields with defaults plus some conditionals (e.g. if flag X is set, do something different). This is working ok, but is no longer sufficient, not flexible enough etc.

      What I need:
      Tests, which are able to create/modify contracts in GUI. Contracts may be complex, so I need to specify e.g. some base contract with predefined parameters and have ability to e.g. specify different set of products, discounts etc.
      The purpose of these tests is functional/acceptance/integration/regression testing.

      Approach:
      I see two possible approaches:
      1. Specify target state, let test do the work.
      2. Specify operations, let dumb test to execute them.

      Questions:
      1. Did you solve similar problem?
      2. What was the essence of your approach?
      3. What patterns emerged from your solution?
      4. Any other ideas/observations?

      Thanks.
      Oleg
    • D.André Dhondt
      ... Why do you need tests that hit the GUI directly? Is your business logic in a testable layer? Can you imagine how to separate the functional tests from
      Message 2 of 8 , Jan 19, 2010
      • 0 Attachment
        On Tue, Jan 19, 2010 at 2:44 PM, strazhce <infobox.oleg@...> wrote:

        > >What I need: Tests, which are able to create/modify contracts in GUI
        >

        Why do you need tests that hit the GUI directly? Is your business logic in
        a testable layer? Can you imagine how to separate the functional tests from
        the integration tests?


        --
        D. André Dhondt
        http://dhondtsayitsagile.blogspot.com/

        Support low-cost conferences -- http://agiletour.org/
        If you're in the area, join Agile Philly http://www.AgilePhilly.com
        Mentor/be mentored: the Agile Skills Project
        https://sites.google.com/site/agileskillsprojectwiki/


        [Non-text portions of this message have been removed]
      • Curtis Cooley
        ... patterns, so even if you decide to roll your own framework, you ll get some good info. At a high level, you want a way to specify test data, FitNesse uses
        Message 3 of 8 , Jan 19, 2010
        • 0 Attachment
          On Tue, Jan 19, 2010 at 5:44 AM, strazhce <infobox.oleg@...> wrote:

          >
          >
          > Hi.
          >
          > I'm trying to figure out the proper structure of tests for our project. I
          > have several ideas and I wonder, if you know some tested patterns for doing
          > that.
          >
          > My situation:
          > I have a web application, which manages contracts. There are few type of
          > agendas - create contract, change contract, cancel contract etc. Every
          > agenda has a wizard-like sequence of screens, each one may have some
          > subscreens tied to button or some actions based on buttons (like "add item"
          > etc.).
          > We have tests, which are pretty monolithic - e.g.
          > CreateContractForProductX, which has most parameters as fields with defaults
          > plus some conditionals (e.g. if flag X is set, do something different). This
          > is working ok, but is no longer sufficient, not flexible enough etc.
          >
          > What I need:
          > Tests, which are able to create/modify contracts in GUI. Contracts may be
          > complex, so I need to specify e.g. some base contract with predefined
          > parameters and have ability to e.g. specify different set of products,
          > discounts etc.
          > The purpose of these tests is functional/acceptance/integration/regression
          > testing.
          >
          > Approach:
          > I see two possible approaches:
          > 1. Specify target state, let test do the work.
          > 2. Specify operations, let dumb test to execute them.
          >
          > Questions:
          > 1. Did you solve similar problem?
          > 2. What was the essence of your approach?
          > 3. What patterns emerged from your solution?
          > 4. Any other ideas/observations?
          >
          > Thanks.
          > Oleg
          >
          > I'd suggest taking a look at FitNesse or Cucumber. At http://fitnesse.orgthere is a ton of written documentation on testing and acceptance test
          patterns, so even if you decide to roll your own framework, you'll get some
          good info.

          At a high level, you want a way to specify test data, FitNesse uses html
          tables. Then you want a way to feed that data into your test fixtures. Then
          you need a way to compare the output to the expected output. Again FitNesse
          uses the same html tables it uses for input. Also, find a way to exercise
          your application without going through the GUI. If you are willing to write
          or rewrite a bunch of acceptance tests, also take the time to seperate your
          GUI layer from your application layer so you can test easier.

          --
          Curtis Cooley
          curtis.cooley@...
          home:http://curtiscooley.com
          blog:http://ponderingobjectorienteddesign.blogspot.com
          ===============
          Leadership is a potent combination of strategy and character. But if you
          must be without one, be without the strategy.
          -- H. Norman Schwarzkopf


          [Non-text portions of this message have been removed]
        • strazhce
          Hi, Andre. Very good questions, I m considering this as well. ... Why - I don t know how to do that through controller only properly. I think we could pull
          Message 4 of 8 , Jan 19, 2010
          • 0 Attachment
            Hi, Andre.

            Very good questions, I'm considering this as well.

            --- In extremeprogramming@yahoogroups.com, D.André Dhondt <d.andre.dhondt@...> wrote:
            >
            > Why do you need tests that hit the GUI directly? Is your business logic in
            > a testable layer? Can you imagine how to separate the functional tests from
            > the integration tests?

            Why - I don't know how to do that through controller only properly.

            I think we could pull strings of our M/C of MVC, but it would be very hard :-(
            - controller API is very advanced, with strict separation of View from MC, but also complex at the same time
            - certain validations/controller API calls are tied to GUI transitions and events generated from there - we would have to fire those events manually in tests
            - our current skills and application related utilities for unit testing are not very good. We would have to change testing paradigm of the team from GUI oriented testing to something else, that's hard. That's a large topic for itself.

            Is it common to create another layer around controller for unit/integration testing purposes, to simplify things? Something similar to what I want for GUI testing? I don't see another way around those complexities above. But that's why I post here ;-)

            Oleg
          • Curtis Cooley
            ... lost count of the times and number of agilists I ve heard advise making the test the first interface and the GUI the second. That does not help you now,
            Message 5 of 8 , Jan 19, 2010
            • 0 Attachment
              On Tue, Jan 19, 2010 at 3:07 PM, strazhce <infobox.oleg@...> wrote:

              > Is it common to create another layer around controller for
              > unit/integration testing purposes, to simplify things? Something similar to
              > what I want for GUI testing? I don't see another way around those
              > complexities above. But that's why I post here ;-)
              >
              > Not only common but the recommended way to do this kind of testing. I've
              lost count of the times and number of agilists I've heard advise making the
              test the first interface and the GUI the second.

              That does not help you now, sadly, since your GUI is already built, but
              incrementally attacking that complex controller interface to make it more
              testable might also lead to some other benefits as well.

              Automated Monkey Testing (Robotest for example) is so fragile and unruly
              that I rarely see it mentioned as an option, let alone put into production
              enough times by the same person/people to have reliable and useful patterns
              emerge. Where as FitNesse for example has been used over and over by the
              same groups of people that good patterns are starting to emerge.

              --
              Curtis Cooley
              curtis.cooley@...
              home:http://curtiscooley.com
              blog:http://ponderingobjectorienteddesign.blogspot.com
              ===============
              Leadership is a potent combination of strategy and character. But if you
              must be without one, be without the strategy.
              -- H. Norman Schwarzkopf


              [Non-text portions of this message have been removed]
            • George Dinwiddie
              Oleg, I suggest extracting the business logic into a separate tier. It then becomes the model for the MVC pattern. See http://bit.ly/6K2Nhp for more
              Message 6 of 8 , Jan 19, 2010
              • 0 Attachment
                Oleg,

                I suggest extracting the business logic into a separate tier. It then
                becomes the model for the MVC pattern. See http://bit.ly/6K2Nhp for
                more discussion on this.

                This lets you test the business rules directly against this layer. Only
                the navigation through the app needs to be tested through the GUI.

                - George

                strazhce wrote:
                > Hi, Andre.
                >
                > Very good questions, I'm considering this as well.
                >
                > --- In extremeprogramming@yahoogroups.com, D.André Dhondt <d.andre.dhondt@...> wrote:
                >> Why do you need tests that hit the GUI directly? Is your business logic in
                >> a testable layer? Can you imagine how to separate the functional tests from
                >> the integration tests?
                >
                > Why - I don't know how to do that through controller only properly.
                >
                > I think we could pull strings of our M/C of MVC, but it would be very hard :-(
                > - controller API is very advanced, with strict separation of View
                > from MC, but also complex at the same time
                > - certain validations/controller API calls are tied to GUI
                > transitions and events generated from there - we would have to fire
                > those events manually in tests
                > - our current skills and application related utilities for unit
                > testing are not very good. We would have to change testing paradigm
                > of the team from GUI oriented testing to something else, that's hard.
                > That's a large topic for itself.
                >
                > Is it common to create another layer around controller for
                > unit/integration testing purposes, to simplify things? Something
                > similar to what I want for GUI testing? I don't see another way
                > around those complexities above. But that's why I post here ;-)


                --
                ----------------------------------------------------------------------
                * George Dinwiddie * http://blog.gdinwiddie.com
                Software Development http://www.idiacomputing.com
                Consultant and Coach http://www.agilemaryland.org
                ----------------------------------------------------------------------
              • D.André Dhondt
                ... Anything of value has its price ;) Take baby steps, hit just a flaky test, or just one slow/expensive test, or even a test that could be written easily to
                Message 7 of 8 , Jan 19, 2010
                • 0 Attachment
                  On Wed, Jan 20, 2010 at 12:07 AM, strazhce <infobox.oleg@...> wrote:

                  > I think we could pull strings of our M/C of MVC, but it would be very hard
                  > :-(
                  >
                  Anything of value has its price ;)
                  Take baby steps, hit just a flaky test, or just one slow/expensive test, or
                  even a test that could be written easily to find out if decoupling it from
                  the gui helps. Do something that is important for the current iteration,
                  and slowly the team can re-align the tests to the new vision of testing, or
                  improve that vision, then re-align...


                  > - certain validations/controller API calls are tied to GUI transitions and
                  > events generated from there - we would have to fire those events manually in
                  > tests
                  >
                  I don't have a problem with firing events directly from test code. There is
                  some risk that the event isn't wired up to the GUI, but regressions there
                  are rare (in my experience) so I don't bother automating gui event tests.
                  Instead, I test the behavior of the event itself, since that's what I care
                  about.


                  > - our current skills and application related utilities for unit testing are
                  > not very good. We would have to change testing paradigm of the team from GUI
                  > oriented testing to something else, that's hard. That's a large topic for
                  > itself.
                  >
                  It is, but instead of changing the whole team, reduce scope... just try one
                  test, show it to people, ask for their help, discuss, weigh the pros and
                  cons, and make it better... or attack the next test, together. Bit by bit,
                  the team's vision of the testing paradigm will change all by itself.



                  --
                  D. André Dhondt
                  http://dhondtsayitsagile.blogspot.com/

                  Support low-cost conferences -- http://agiletour.org/
                  If you're in the area, join Agile Philly http://www.AgilePhilly.com
                  Mentor/be mentored: the Agile Skills Project
                  https://sites.google.com/site/agileskillsprojectwiki/


                  [Non-text portions of this message have been removed]
                • JeffGrigg
                  In my experience of retrofitting tests into legacy code, I find it helpful to think of the system having a stack of layers like this: 0. The user 1. Views 2.
                  Message 8 of 8 , Jan 20, 2010
                  • 0 Attachment
                    In my experience of retrofitting tests into legacy code, I find it helpful to think of the system having a stack of layers like this:

                    0. The user
                    1. Views
                    2. Presenter/Controller
                    3. Business Logic
                    4. Data Access Objects
                    5. Database / data store

                    There are useful tools, like Selenium, that will test the whole stack. But they're slow and brittle, as the system changes.

                    So I end up writing mocks/fakes for the top and bottom layers (1. Views and 5. Database) and testing everything in between (2 thru 4).

                    Use weak Singletons (instances replaceable by test code) and possibly factory objects stored as Singletons to mock the lowest levels -- particularly when they're called through static functions. It's unpleasant, but it gets the tests working. And you can refactor the Singleton-ness out later, as needed.


                    --- "strazhce" <infobox.oleg@...> wrote:
                    > - controller API is very advanced, with strict separation of
                    > View from MC, but also complex at the same time

                    This sounds like good news: If well separated, then you should be able to mock the view layer easily.

                    > - certain validations/controller API calls are tied to GUI
                    > transitions and events generated from there - we would
                    > have to fire those events manually in tests

                    Yes, you'll have to simulate GUI events from your tests.

                    > - our current skills and application related utilities for
                    > unit testing are not very good. We would have to change
                    > testing paradigm of the team from GUI oriented testing
                    > to something else, that's hard. [...]

                    Look at it as on opportunity to learn. And don't think you're limited to just one paradigm: You can have unit tests, integration tests and acceptance tests all working together to test a single system! I'll often have mock-based integration tests that display no GUI and do not hit the database, and also run Selenium tests against the full stack, with GUI and test database. You can do more than one thing. And during transitions, you'll have to.

                    > Is it common to create another layer around controller for
                    > unit/integration testing purposes, to simplify things?

                    I say mock the view. Use the real controller and business logic. Mock the view and the database.

                    Thank you for the questions! We're all glad to help!
                    - jeff
                  Your message has been successfully submitted and would be delivered to recipients shortly.