Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Adventures in C#: How Do We Do a Test That We Can't Do?

Expand Messages
  • Daniel Berlinger
    ... Ron, Would it be wrong to write some notes that explain your thinking of the moment (you re on a role, you know just how to do it). Or is that almost as
    Message 1 of 12 , Nov 1, 2002
    • 0 Attachment
      > Wait! Wait, wait, wait!
      >
      > > http://www.xprogramming.com/xpmag/acsGUItesting.htm#N332
      >
      Ron,

      Would it be wrong to write some notes that explain your thinking "of the moment" (you're on a role, you know just how to do it). Or is that almost as bad as coding with the fantasy of the future?

      Thanks,

      d.

      PS I use the terms "bad" and "wrong" in the context of the lesson.
    • Ron Jeffries
      ... I guess you re asking if as a developer I might write something down to be sure that I remember how to do it? I might well do that. On the other hand, I
      Message 2 of 12 , Nov 1, 2002
      • 0 Attachment
        On Friday, November 1, 2002, at 11:10:01 AM, Daniel Berlinger wrote:

        >> Wait! Wait, wait, wait!
        >>
        >> > http://www.xprogramming.com/xpmag/acsGUItesting.htm#N332
        >>
        > Ron,

        > Would it be wrong to write some notes that explain your thinking "of the moment" (you're on a role, you know
        > just how to do it). Or is that almost as bad as coding with the fantasy of the future?

        I guess you're asking if as a developer I might write something down
        to be sure that I remember how to do it? I might well do that.

        On the other hand, I like to live on the edge, and I enjoy learning
        new things. So I don't mind forgetting stuff, because it's so exciting
        to discover it later. ;->

        Ron Jeffries
        www.XProgramming.com
        "How do I get to XP?" "Practice, man, practice."
      • Charlie Poole
        Ron, I want to think a bit about what you re doing here, but in the spirit of just telling it as it happens, here s what s in my head right now. What is Ron
        Message 3 of 12 , Nov 1, 2002
        • 0 Attachment
          Ron,

          I want to think a bit about what you're doing here, but in the spirit
          of just telling it as it happens, here's what's in my head right now.

          What is Ron testing? He's already tested that the model does what
          it should do when it's methods are called correctly, so what's left?

          Here's a first cut.
          * Test that altS causes the model's correct method to be called correctly.

          Hmmm... That certainly covers it all in one shot, but it seemed hard
          to test. What could possibly go wrong?

          Candidates:
          1. The Windows operating system might ignore the accelerator
          2. The Accelerator might not be on the menu item
          3. The menu Item might not be on the form and it's main menu
          4. The menu Item might not generate a click event when clicked
          5. The click event might not be connected to the proper handler
          6. The handler might not call the proper model method.

          I put items 1 and 4 in as things that probably don't need to be
          tested. Of course, in my own code, it sometimes seems as if one
          of these things has happened, but it usually turns out that
          I've done something wrong myself.

          So how hard is it to test for the other items?

          Items 2 and 3 can be looked at by instantiating the form and
          examining the menu and menu item. Something like the following
          untested code:

          MyForm form = new MyForm(...);
          Assert( form.Menu.MenuItems.Contains( form.someMenuItem ) );
          Assert( form.someMenuItem.Mnemonic == 'S' );
          etc.

          Items 5 and 6 could be tested by a mock model or using the
          actual model as you've done. It strikes me that a mock model
          that did nothing but report on how it had been called and what
          it's arguments were would make isolated testing of the ui a
          lot easier. So...

          MyModel model = new MyModel(...);
          MyForm form = new MyForm( model );

          // Init form and model as needed

          form.MenuItem.PerformClick();
          Assert( model.SomeMethodWasCalled );
          AssertEquals( "somestring", model.SomeMethodParametersPassed );

          Of course, the same thing can be done with the real model by checking
          that the state has been appropriately modified.

          Like I said, just an initial set of thoughts...

          Charlie Poole
          cpoole@...
          www.pooleconsulting.com
          www.charliepoole.org



          > -----Original Message-----
          > From: Ron Jeffries [mailto:ronjeffries@...]
          > Sent: Friday, November 01, 2002 6:46 AM
          > To: extremeprogramming@yahoogroups.com
          > Subject: Re: [XP] Adventures in C#: How Do We Do a Test That We Can't
          > Do?
          >
          >
          > Around Friday, November 1, 2002, 7:01:05 AM, Ron Jeffries wrote:
          >
          > Wait! Wait, wait, wait!
          >
          > > http://www.xprogramming.com/xpmag/acsGUItesting.htm#N332
          >
          > It's cool writing a book where if you screw up it's just part of the
          > story ...
          >
          > Ron Jeffries
          > www.XProgramming.com
          > I could be wrong, but I'm not. --Eagles, Victim of Love
          >
          >
          > To Post a message, send it to: extremeprogramming@...
          >
          > To Unsubscribe, send a blank message to:
          > extremeprogramming-unsubscribe@...
          >
          > ad-free courtesy of objectmentor.com
          >
          > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
          >
          >
          >
          >
        • Kevin Lawrence
          From: Charlie Poole ... I was going to say something similar to Charlie but I got distracted with an essay about estimating. So
          Message 4 of 12 , Nov 1, 2002
          • 0 Attachment
            From: "Charlie Poole" <cpoole@...>
            >
            > Items 2 and 3 can be looked at by instantiating the form and
            > examining the menu and menu item. Something like the following
            > untested code:
            >
            > MyForm form = new MyForm(...);
            > Assert( form.Menu.MenuItems.Contains( form.someMenuItem ) );
            > Assert( form.someMenuItem.Mnemonic == 'S' );
            > etc.
            >


            I was going to say something similar to Charlie but I got distracted with an
            essay about estimating. So I'll delete my reply and I'll add to Charlie's.

            Here's what I do.

            I assert that the shortcut is correct
            AssertEquals(Shortcut.CtrlP, menu.Shortcut); - or whatever

            I assert that the label is right
            AssertEquals("Insert &Section", menu.Text);

            I assert that the menu does what it is supposed to when I click it
            menu.PerformClick();
            AssertEquals("<p></p>", textbox.Text);
            AssertEquals(3, textbox.SelectionStart);

            I assume that Bill Gates & Co have made the little ampersand thingie and the
            shortcut key work right so I don't test that.

            If I wanted to implement your script-driven acceptance tests, I might be
            tempted to write them in terms of menu labels rather than shortcut keys (and
            assume that they are equivalent).

            *input
            <p>whatever</p>
            *Edit|Insert Section
            *output
            etc etc etc

            That would make it supremely easy to match script commands with menu
            objects.


            > Items 5 and 6 could be tested by a mock model or using the
            > actual model as you've done. It strikes me that a mock model
            > that did nothing but report on how it had been called and what
            > it's arguments were would make isolated testing of the ui a
            > lot easier. So...
            >
            > MyModel model = new MyModel(...);
            > MyForm form = new MyForm( model );
            >
            > // Init form and model as needed
            >
            > form.MenuItem.PerformClick();
            > Assert( model.SomeMethodWasCalled );
            > AssertEquals( "somestring", model.SomeMethodParametersPassed );
            >
            > Of course, the same thing can be done with the real model by checking
            > that the state has been appropriately modified.
            >

            In my product, I mocked the model like this very early on and - now that
            things are a little more complex - I really wish I hadn't. I have had a few
            little bugs slip down the crack between testing the model and testing the UI
            and - now that my real model has a lot of internal magic going on, I find
            myself having to fake that magic in the mock or in my tests. I am
            duplicating logic between the model and the mock model when it would have
            been much simpler to use the real model in the first place.

            Lesson learned : Don't use a mock when it's easy enough to use the real
            thing. I think somebody else said that already - but I had to learn it for
            myself :-(


            Kevin

            Diamond Sky Software
            www.diamond-sky.com
          • Ron Jeffries
            ... I haven t replied to charlie, so I ll reply to you. ;- ... I m doing this one ... ... part of this one ... ... and all of this one, with the one test that
            Message 5 of 12 , Nov 1, 2002
            • 0 Attachment
              On Friday, November 1, 2002, at 3:22:32 PM, Kevin Lawrence wrote:

              > From: "Charlie Poole" <cpoole@...>
              >>
              >> Items 2 and 3 can be looked at by instantiating the form and
              >> examining the menu and menu item. Something like the following
              >> untested code:
              >>
              >> MyForm form = new MyForm(...);
              >> Assert( form.Menu.MenuItems.Contains( form.someMenuItem ) );
              >> Assert( form.someMenuItem.Mnemonic == 'S' );
              >> etc.
              >>


              > I was going to say something similar to Charlie but I got distracted with an
              > essay about estimating. So I'll delete my reply and I'll add to Charlie's.

              I haven't replied to charlie, so I'll reply to you. ;->

              > Here's what I do.

              > I assert that the shortcut is correct
              > AssertEquals(Shortcut.CtrlP, menu.Shortcut); - or whatever

              I'm doing this one ...

              > I assert that the label is right
              > AssertEquals("Insert &Section", menu.Text);

              part of this one ...

              > I assert that the menu does what it is supposed to when I click it
              > menu.PerformClick();
              > AssertEquals("<p></p>", textbox.Text);
              > AssertEquals(3, textbox.SelectionStart);

              and all of this one, with the one test that runs &P. It just turned
              out that way. I'm not inclined to test the menu labels but maybe I'll
              learn that I should.

              > I assume that Bill Gates & Co have made the little ampersand thingie and the
              > shortcut key work right so I don't test that.

              Yes, me too.

              > If I wanted to implement your script-driven acceptance tests, I might be
              > tempted to write them in terms of menu labels rather than shortcut keys (and
              > assume that they are equivalent).

              > *input
              > <p>whatever</p>
              > *Edit|Insert Section
              > *output
              > etc etc etc

              > That would make it supremely easy to match script commands with menu
              > objects.

              Yes. I predict that it will turn out rather close to that way. The
              menu command will be something like *menu any string at all, and will
              be implemented by looking for a menu with that string in it, and
              calling it.


              ...
              >>
              >> Of course, the same thing can be done with the real model by checking
              >> that the state has been appropriately modified.
              >>

              > In my product, I mocked the model like this very early on and - now that
              > things are a little more complex - I really wish I hadn't. I have had a few
              > little bugs slip down the crack between testing the model and testing the UI
              > and - now that my real model has a lot of internal magic going on, I find
              > myself having to fake that magic in the mock or in my tests. I am
              > duplicating logic between the model and the mock model when it would have
              > been much simpler to use the real model in the first place.

              > Lesson learned : Don't use a mock when it's easy enough to use the real
              > thing. I think somebody else said that already - but I had to learn it for
              > myself :-(

              That's kind of where I'm at, though I can't say why ...

              Thanks,

              Ron Jeffries
              www.XProgramming.com
              First they ignore you, then they laugh at you, then they fight you, then you win.
              - Gandhi
            • Ron Jeffries
              ... Agreed ... ... This kind of test doesn t serve me, somehow. Partly, it s because it s tieing me down to someMenuItem as a name. Partly, there s just
              Message 6 of 12 , Nov 1, 2002
              • 0 Attachment
                On Friday, November 1, 2002, at 2:54:07 PM, Charlie Poole wrote:

                > Here's a first cut.
                > * Test that altS causes the model's correct method to be called correctly.

                > Hmmm... That certainly covers it all in one shot, but it seemed hard
                > to test. What could possibly go wrong?

                > Candidates:
                > 1. The Windows operating system might ignore the accelerator
                > 2. The Accelerator might not be on the menu item
                > 3. The menu Item might not be on the form and it's main menu
                > 4. The menu Item might not generate a click event when clicked
                > 5. The click event might not be connected to the proper handler
                > 6. The handler might not call the proper model method.

                > I put items 1 and 4 in as things that probably don't need to be
                > tested. Of course, in my own code, it sometimes seems as if one
                > of these things has happened, but it usually turns out that
                > I've done something wrong myself.

                Agreed ...

                > So how hard is it to test for the other items?

                > Items 2 and 3 can be looked at by instantiating the form and
                > examining the menu and menu item. Something like the following
                > untested code:

                > MyForm form = new MyForm(...);
                > Assert( form.Menu.MenuItems.Contains( form.someMenuItem ) );
                > Assert( form.someMenuItem.Mnemonic == 'S' );
                > etc.

                This kind of test doesn't serve me, somehow. Partly, it's because it's
                tieing me down to "someMenuItem" as a name. Partly, there's just
                something to rote or obvious about it ... ah. It doesn't drive me to
                build new functionality in the system, just to put in a menu item. It
                doesn't drive design either. Something like that ...

                > Items 5 and 6 could be tested by a mock model or using the
                > actual model as you've done. It strikes me that a mock model
                > that did nothing but report on how it had been called and what
                > it's arguments were would make isolated testing of the ui a
                > lot easier. So...

                > MyModel model = new MyModel(...);
                > MyForm form = new MyForm( model );

                > // Init form and model as needed

                > form.MenuItem.PerformClick();
                > Assert( model.SomeMethodWasCalled );
                > AssertEquals( "somestring", model.SomeMethodParametersPassed );

                > Of course, the same thing can be done with the real model by checking
                > that the state has been appropriately modified.

                Yes. And it seems to me that the form can be out of sync with the
                mock. For example, the handlers might have their names reversed and
                exist but do exactly the wrong thing ...

                > Like I said, just an initial set of thoughts...

                And interesting ones at that. Thanks,

                Ron Jeffries
                www.XProgramming.com
                The rules are ways of thinking, not ways to avoid thinking.
              • Kevin Lawrence
                From: Ron Jeffries ... You made me question why I do this in my own tests... I decided that it s somewhat analogous to your previous
                Message 7 of 12 , Nov 1, 2002
                • 0 Attachment
                  From: "Ron Jeffries" <ronjeffries@...>
                  >
                  > > MyForm form = new MyForm(...);
                  > > Assert( form.Menu.MenuItems.Contains( form.someMenuItem ) );
                  > > Assert( form.someMenuItem.Mnemonic == 'S' );
                  > > etc.
                  >
                  > This kind of test doesn't serve me, somehow. Partly, it's because it's
                  > tieing me down to "someMenuItem" as a name. Partly, there's just
                  > something to rote or obvious about it ... ah. It doesn't drive me to
                  > build new functionality in the system, just to put in a menu item. It
                  > doesn't drive design either. Something like that ...
                  >

                  You made me question why I do this in my own tests...

                  I decided that it's somewhat analogous to your previous dilemma about adding
                  the line of code that makes the thing scroll correctly without tests.

                  The test doesn't drive the design and it doesn't drive you to put new
                  functionality in ... but it does stop you from accidently deleting that line
                  during a later refactoring. You wouldn't want to be in a situation where the
                  story worked for everyone but the users ;-)

                  Sometimes a test is just a test.

                  Kevin
                • Ron Jeffries
                  ... Yes. I prefer a test that does both. I m not sure that I have them now, and I surely don t always get them. But that s the aesthetic that s driving me. Ron
                  Message 8 of 12 , Nov 1, 2002
                  • 0 Attachment
                    On Friday, November 1, 2002, at 5:52:45 PM, Kevin Lawrence wrote:

                    > The test doesn't drive the design and it doesn't drive you to put new
                    > functionality in ... but it does stop you from accidently deleting that line
                    > during a later refactoring. You wouldn't want to be in a situation where the
                    > story worked for everyone but the users ;-)

                    > Sometimes a test is just a test.

                    Yes. I prefer a test that does both. I'm not sure that I have them
                    now, and I surely don't always get them. But that's the aesthetic
                    that's driving me.

                    Ron Jeffries
                    www.XProgramming.com
                    Bang, bang, Jeffries' silver hammer came down upon their heads ...
                  • Alex Chaffee / Purple Technology
                    ... Oh, that s nice. I ve been trying lately to make the leap from mostly-test-first to fully-test-first and I find it s difficult to go back to the test code
                    Message 9 of 12 , Nov 3, 2002
                    • 0 Attachment
                      On Fri, Nov 01, 2002 at 02:52:45PM -0800, Kevin Lawrence wrote:
                      > From: "Ron Jeffries" <ronjeffries@...>
                      > >
                      > > > MyForm form = new MyForm(...);
                      > > > Assert( form.Menu.MenuItems.Contains( form.someMenuItem ) );
                      > > > Assert( form.someMenuItem.Mnemonic == 'S' );
                      > > > etc.
                      > >
                      > > This kind of test doesn't serve me, somehow.
                      > > ... ah. It doesn't drive me to
                      > > build new functionality in the system, just to put in a menu item. It
                      > > doesn't drive design either. Something like that ...
                      > >
                      >
                      > You made me question why I do this in my own tests...
                      >
                      > I decided that it's somewhat analogous to your previous dilemma about adding
                      > the line of code that makes the thing scroll correctly without tests.
                      >
                      > The test doesn't drive the design and it doesn't drive you to put new
                      > functionality in ... but it does stop you from accidently deleting that line
                      > during a later refactoring. You wouldn't want to be in a situation where the
                      > story worked for everyone but the users ;-)
                      >
                      > Sometimes a test is just a test.

                      Oh, that's nice. I've been trying lately to make the leap from
                      mostly-test-first to fully-test-first and I find it's difficult to go
                      back to the test code in order to add a few lines of test code when I
                      already *know* what the implementation code is going to be -- and it's
                      often going to have fewer lines of code than the test).

                      "Sometimes a test is just a test" captures the spirit of how I can
                      convince myself to go back and add the brief test code for writing
                      literally every line of code test-first. It's not because I need a
                      test to drive the design. It's because I need a test!

                      (Still not sure if it's worth doing all the tim... but at least I'm
                      trying to learn how to do it before I make up my mind about that.)

                      - A



                      --
                      Alex Chaffee mailto:alex@...
                      jGuru - Java News and FAQs http://www.jguru.com/alex/
                      Creator of Gamelan http://www.gamelan.com/
                      Founder of Purple Technology http://www.purpletech.com/
                      Curator of Stinky Art Collective http://www.stinky.com/
                    • Ron Jeffries
                      ... This is good. I ve been pushing as hard as I can on the practices since I got into XP and I have found that it teaches me things, builds good habits, and
                      Message 10 of 12 , Nov 4, 2002
                      • 0 Attachment
                        On Monday, November 4, 2002, at 12:16:43 AM, Alex Chaffee / Purple Technology wrote:

                        > On Fri, Nov 01, 2002 at 02:52:45PM -0800, Kevin Lawrence wrote:
                        >> From: "Ron Jeffries" <ronjeffries@...>
                        >> >
                        >> > > MyForm form = new MyForm(...);
                        >> > > Assert( form.Menu.MenuItems.Contains( form.someMenuItem ) );
                        >> > > Assert( form.someMenuItem.Mnemonic == 'S' );
                        >> > > etc.
                        >> >
                        >> > This kind of test doesn't serve me, somehow.
                        >> > ... ah. It doesn't drive me to
                        >> > build new functionality in the system, just to put in a menu item. It
                        >> > doesn't drive design either. Something like that ...
                        >> >
                        >>
                        >> You made me question why I do this in my own tests...
                        >>
                        >> I decided that it's somewhat analogous to your previous dilemma about adding
                        >> the line of code that makes the thing scroll correctly without tests.
                        >>
                        >> The test doesn't drive the design and it doesn't drive you to put new
                        >> functionality in ... but it does stop you from accidently deleting that line
                        >> during a later refactoring. You wouldn't want to be in a situation where the
                        >> story worked for everyone but the users ;-)
                        >>
                        >> Sometimes a test is just a test.

                        > Oh, that's nice. I've been trying lately to make the leap from
                        > mostly-test-first to fully-test-first and I find it's difficult to go
                        > back to the test code in order to add a few lines of test code when I
                        > already *know* what the implementation code is going to be -- and it's
                        > often going to have fewer lines of code than the test).

                        > "Sometimes a test is just a test" captures the spirit of how I can
                        > convince myself to go back and add the brief test code for writing
                        > literally every line of code test-first. It's not because I need a
                        > test to drive the design. It's because I need a test!

                        > (Still not sure if it's worth doing all the tim... but at least I'm
                        > trying to learn how to do it before I make up my mind about that.)

                        This is good. I've been pushing as hard as I can on the practices
                        since I got into XP and I have found that it teaches me things, builds
                        good habits, and presents interesting and fun discoveries.

                        I wanted to be clear about my comments above about this kind of test.
                        Like you and Kevin, I'll do them if I feel I need them and can't think
                        of anything better. A safety test is better than no test at all.

                        The kind of test that I /prefer/ is the kind I describe above, where
                        the test drives me to do something, and to learn something. Those are
                        the tests, for me, where the riches lie.

                        Ron Jeffries
                        www.XProgramming.com
                        Sorry about your cow ... I didn't know she was sacred.
                      Your message has been successfully submitted and would be delivered to recipients shortly.