Loading ...
Sorry, an error occurred while loading the content.

New Article: Shooting for Stories

Expand Messages
  • Ron Jeffries
    Shooting for Stories Ron Jeffries 12/06/2006 We ve been working for almost two full days now, and it seemed time to refine some stories. We found some
    Message 1 of 25 , Dec 6, 2006
    • 0 Attachment
      Shooting for Stories
      Ron Jeffries
      12/06/2006

      We've been working for almost two full days now, and it seemed
      time to refine some stories. We found some conflicts between
      ourselves as programmers, ourselves as customers, and ourselves as
      advisors to projects.

      http://www.xprogramming.com/xpmag/dbcShootingForStories.htm


      Ron Jeffries
      www.XProgramming.com
      Prediction is very difficult, especially if it's about the future. -- Niels Bohr
    • John A. De Goes
      Hi Ron, ... Hmmm, sounds familiar. Didn t I suggest there were cases like this out there, some 3-4 months ago? Nice to see you ve finally come around. :-) I
      Message 2 of 25 , Dec 6, 2006
      • 0 Attachment
        Hi Ron,

        > This test reads the big BMP file of the real target. It asserts that
        > the CenterOfMass is (123,281). You can be sure that we didn't hand
        > calculate that number. We ran the test, looked at the answer, looked
        > at the picture, marked where that point was, and accepted the result.

        Hmmm, sounds familiar. Didn't I suggest there were cases like this out
        there, some 3-4 months ago? Nice to see you've finally come around. :-)

        I look forward to the next installment.

        Regards,

        John

        Quoting Ron Jeffries <ronjeffries@...>:

        > Shooting for Stories
        > Ron Jeffries
        > 12/06/2006
        >
        > We've been working for almost two full days now, and it seemed
        > time to refine some stories. We found some conflicts between
        > ourselves as programmers, ourselves as customers, and ourselves as
        > advisors to projects.
        >
        > http://www.xprogramming.com/xpmag/dbcShootingForStories.htm
        >
        >
        > Ron Jeffries
        > www.XProgramming.com
        > Prediction is very difficult, especially if it's about the future.
        > -- Niels Bohr
        >
        >
      • Ron Jeffries
        Hello, John. On Wednesday, December 6, 2006, at 9:39:58 AM, you ... I m glad to see you re reading the articles, John. And I ll update that one to report a
        Message 3 of 25 , Dec 6, 2006
        • 0 Attachment
          Hello, John. On Wednesday, December 6, 2006, at 9:39:58 AM, you
          wrote:

          >> This test reads the big BMP file of the real target. It asserts that
          >> the CenterOfMass is (123,281). You can be sure that we didn't hand
          >> calculate that number. We ran the test, looked at the answer, looked
          >> at the picture, marked where that point was, and accepted the result.

          > Hmmm, sounds familiar. Didn't I suggest there were cases like this out
          > there, some 3-4 months ago? Nice to see you've finally come around. :-)

          I'm glad to see you're reading the articles, John. And I'll update
          that one to report a bit more of the conversation.

          Golden Master tests have been around a long time, and they have some
          drawbacks.

          > I look forward to the next installment.

          I'm working on it now, here at the Stonehouse Cafe ...

          Thanks,

          Ron Jeffries
          www.XProgramming.com
          Reason is and ought only to be the slave of the passions. -- David Hume
        • Donald Roby
          ... (From the article...) ... If your test data is in your project, it must be the same on both machines, relatively speaking.... You can simply specify a
          Message 4 of 25 , Dec 6, 2006
          • 0 Attachment
            Ron Jeffries wrote:

            >Shooting for Stories
            >Ron Jeffries
            >12/06/2006
            >
            > We've been working for almost two full days now, and it seemed
            > time to refine some stories. We found some conflicts between
            > ourselves as programmers, ourselves as customers, and ourselves as
            > advisors to projects.
            >
            > http://www.xprogramming.com/xpmag/dbcShootingForStories.htm
            >
            >
            >
            (From the article...)

            > 1. We have some sample files that our tests use. Since our folder
            > layouts are different, the naive approach of putting the file
            > paths in the tests means that one of us will edit the paths to
            > make the tests run. Then when he checks in, they break for the
            > other guy. There is probably some standard way in Eclipse / Java
            > to set up a kind of path or something that could be different on
            > our two systems, but if there is we haven't been able to search
            > it out. If you know a good way, an email note will be helpful.
            > Thanks!
            >
            If your test data is in your project, it must be the same on both
            machines, relatively speaking....

            You can simply specify a relative path. It may not work if you run your
            tests outside Eclipse. unless you cd to the project root directory first.

            Another option is to create a system environment variable for the test
            directory path, set outside Eclipse so that you can set it differently,
            and then have a little bit of Java code get that value and append the
            file name. This might be easier to get working properly under a build
            script that runs outside your IDE. It's more work, but shouldn't be too
            bad.

            --
            Don Roby
          • martijn_meijering
            ... Hello Ron, In your article you write: We still haven t figured out the best way to point to things like files, since Chet organizes his machine a bit
            Message 5 of 25 , Dec 6, 2006
            • 0 Attachment
              --- In extremeprogramming@yahoogroups.com, Ron Jeffries
              <ronjeffries@...> wrote:
              > http://www.xprogramming.com/xpmag/dbcShootingForStories.htm

              Hello Ron,

              In your article you write:

              "We still haven't figured out the best way to point to things like
              files, since Chet organizes his machine a bit differently from mine.
              Anyway, here are the tests::"

              Are you keeping your data files outside your project directory and
              version control system?
            • John A. De Goes
              At my company, because all test resources are stored in the repository, data-dependent tests specify relative paths -- which just work , assuming a developer
              Message 6 of 25 , Dec 6, 2006
              • 0 Attachment
                At my company, because all test resources are stored in the
                repository, data-dependent tests specify relative paths -- which 'just
                work', assuming a developer has checked out the whole repository and
                not just a subfolder.

                It's a slick (and becoming standard) way of dealing with the problem
                of external data.

                Regards,

                John

                Quoting martijn_meijering <mmeijeri@...>:

                > --- In extremeprogramming@yahoogroups.com, Ron Jeffries
                > <ronjeffries@...> wrote:
                >> http://www.xprogramming.com/xpmag/dbcShootingForStories.htm
                >
                > Hello Ron,
                >
                > In your article you write:
                >
                > "We still haven't figured out the best way to point to things like
                > files, since Chet organizes his machine a bit differently from mine.
                > Anyway, here are the tests::"
                >
                > Are you keeping your data files outside your project directory and
                > version control system?
                >
                >
              • Chris Wheeler
                ... Something that has worked for us, for many years, is to use the subst command to assign a drive letter to a directory. If everyone uses the same drive
                Message 7 of 25 , Dec 6, 2006
                • 0 Attachment
                  >
                  > > it out. If you know a good way, an email note will be helpful.
                  > > Thanks!
                  > >
                  >

                  Something that has worked for us, for many years, is to use the subst
                  command to assign a drive letter to a directory. If everyone uses the same
                  drive letter (say, 'x:') then you can always use x:\data (or whatever) on
                  any machine to store your test files, and you can be assured that no matter
                  what developer machine you use, the data is always in the same place.

                  Every developer machine should have a startup batch file that does the subst
                  automagically.

                  Chris.


                  [Non-text portions of this message have been removed]
                • Ron Jeffries
                  Hello, Donald. On Wednesday, December 6, 2006, at 1:36:06 PM, you ... Yes ... so far we haven t figured out quite what the running directory is for the tests
                  Message 8 of 25 , Dec 6, 2006
                  • 0 Attachment
                    Hello, Donald. On Wednesday, December 6, 2006, at 1:36:06 PM, you
                    wrote:

                    > If your test data is in your project, it must be the same on both
                    > machines, relatively speaking....

                    Yes ... so far we haven't figured out quite what the running
                    directory is for the tests inside eclipse, and the fact that we
                    currently handle our workspaces differently may play in there. I
                    suppose I could write a naked file and see where it goes ...

                    > You can simply specify a relative path. It may not work if you run your
                    > tests outside Eclipse. unless you cd to the project root directory first.

                    So far, that's not an issue ...

                    > Another option is to create a system environment variable for the test
                    > directory path, set outside Eclipse so that you can set it differently,
                    > and then have a little bit of Java code get that value and append the
                    > file name. This might be easier to get working properly under a build
                    > script that runs outside your IDE. It's more work, but shouldn't be too
                    > bad.

                    Yes ... it seems easy enough ... we just haven't quite scoped out
                    the odd little system calls it takes to do that kind of things.

                    Carl Manaster sent some example code via email that looks like we
                    can make it work. Just a SMOP.

                    Thanks,

                    Ron Jeffries
                    www.XProgramming.com
                    Agility might be said to be about encountering
                    all the problems so early and so often that the
                    effort to fix them is less than the pain of enduring them.
                  • chet hendrickson
                    Hello John, I think putting the test files in a folder in the project s hierarchy and therefore, under the control of eclipse and subversion is the right way
                    Message 9 of 25 , Dec 6, 2006
                    • 0 Attachment
                      Hello John,

                      I think putting the test files in a folder in the project's hierarchy
                      and therefore, under the control of eclipse and subversion is the
                      right way to handle this. I suspect this is the first thing we will
                      do when Ron and I get together Thursday.

                      Thanks for all the suggestions.

                      chet

                      Wednesday, December 6, 2006, 3:01:10 PM, you wrote:

                      > At my company, because all test resources are stored in the
                      > repository, data-dependent tests specify relative paths -- which 'just
                      > work', assuming a developer has checked out the whole repository and
                      > not just a subfolder.

                      > It's a slick (and becoming standard) way of dealing with the problem
                      > of external data.

                      > Regards,

                      > John

                      > Quoting martijn_meijering <mmeijeri@...>:

                      >> --- In extremeprogramming@yahoogroups.com, Ron Jeffries
                      >> <ronjeffries@...> wrote:
                      >>> http://www.xprogramming.com/xpmag/dbcShootingForStories.htm
                      >>
                      >> Hello Ron,
                      >>
                      >> In your article you write:
                      >>
                      >> "We still haven't figured out the best way to point to things like
                      >> files, since Chet organizes his machine a bit differently from mine.
                      >> Anyway, here are the tests::"
                      >>
                      >> Are you keeping your data files outside your project directory and
                      >> version control system?
                      >>
                      >>





                      --
                      Best regards,
                      chet mailto:lists@...
                    • Ron Jeffries
                      Hello, martijn_meijering. On Wednesday, December 6, 2006, at ... They re currently at the top of the project hierarchy, if I recall. I don t know whether they
                      Message 10 of 25 , Dec 6, 2006
                      • 0 Attachment
                        Hello, martijn_meijering. On Wednesday, December 6, 2006, at
                        1:55:31 PM, you wrote:

                        > --- In extremeprogramming@yahoogroups.com, Ron Jeffries
                        > <ronjeffries@...> wrote:
                        >> http://www.xprogramming.com/xpmag/dbcShootingForStories.htm

                        > Hello Ron,

                        > In your article you write:

                        > "We still haven't figured out the best way to point to things like
                        > files, since Chet organizes his machine a bit differently from mine.
                        > Anyway, here are the tests::"

                        > Are you keeping your data files outside your project directory and
                        > version control system?

                        They're currently at the top of the project hierarchy, if I recall.
                        I don't know whether they are checked in or not, but certainly they
                        should be.

                        We're really at the bottom of the food chain on this, we don't
                        really know what the standard ways of doing this are. We're sure
                        they're out there, just haven't googled them out.

                        That's why I asked our remote pairs. How would you do it?

                        Ron Jeffries
                        www.XProgramming.com
                        There's a difference between righteous anger and just being crabby.
                        --Barbara Richmond
                      • Ron Jeffries
                        Hello, John. On Wednesday, December 6, 2006, at 3:01:10 PM, you ... I must have just not yet correctly guessed the base of the relative path ... Ron Jeffries
                        Message 11 of 25 , Dec 6, 2006
                        • 0 Attachment
                          Hello, John. On Wednesday, December 6, 2006, at 3:01:10 PM, you
                          wrote:

                          > At my company, because all test resources are stored in the
                          > repository, data-dependent tests specify relative paths -- which 'just
                          > work', assuming a developer has checked out the whole repository and
                          > not just a subfolder.

                          > It's a slick (and becoming standard) way of dealing with the problem
                          > of external data.

                          I must have just not yet correctly guessed the base of the relative
                          path ...

                          Ron Jeffries
                          www.XProgramming.com
                          We cannot solve our problems with the same thinking we used when we created them.
                          -- Albert Einstein
                        • Ron Jeffries
                          Hello, Chris. On Wednesday, December 6, 2006, at 3:25:18 PM, you ... Wow, an oldie but a goodie! Ron Jeffries www.XProgramming.com The opinions expressed here
                          Message 12 of 25 , Dec 6, 2006
                          • 0 Attachment
                            Hello, Chris. On Wednesday, December 6, 2006, at 3:25:18 PM, you
                            wrote:

                            > Something that has worked for us, for many years, is to use the subst
                            > command to assign a drive letter to a directory. If everyone uses the same
                            > drive letter (say, 'x:') then you can always use x:\data (or whatever) on
                            > any machine to store your test files, and you can be assured that no matter
                            > what developer machine you use, the data is always in the same place.

                            > Every developer machine should have a startup batch file that does the subst
                            > automagically.

                            Wow, an oldie but a goodie!

                            Ron Jeffries
                            www.XProgramming.com
                            The opinions expressed here /are/ necessarily those of XProgramming.com.
                            But I might change my mind.
                          • John A. De Goes
                            Hi Ron, Here s a good generic project structure (hopefully the spaces will show up): my_project
                            Message 13 of 25 , Dec 6, 2006
                            • 0 Attachment
                              Hi Ron,

                              Here's a good generic project structure (hopefully the spaces will show up):

                              my_project <- Everything's stored here, in root
                              my_ide_project_file.prj
                              src <- Your code
                              com
                              ...
                              lib
                              helperlib.jar
                              ...
                              docs
                              notes.txt
                              ...
                              resources
                              testing
                              test_resource_image012.bmp
                              ...
                              images
                              check_mark_icon01.png
                              ...
                              scripts <- Shortcuts to Ant targets
                              mac-nix
                              deploy-app
                              build-test-app
                              edit-app
                              run-app
                              ...
                              win
                              ...
                              tools <- All tools needed to build app
                              ant
                              compiler
                              editor
                              junit
                              ....
                              temp <- All the dynamically generated stuff
                              classes <- Where the classes are compiled to
                              com
                              ...
                              bin <- Where the jar is placed
                              app.jar
                              testing_wd <- Working directory of tests
                              running_wd <- Working directory of app

                              For this structure, you would reference test data by prefixing it with
                              the concatenated string ".." + File.separator + ".." + File.separator
                              + "resources" + File.separator + "testing" + File.separator. Your Ant
                              targets would take care of specifying the appropriate working
                              directory for executing the application or running the tests.

                              Regards,

                              John

                              Quoting Ron Jeffries <ronjeffries@...>:

                              > Hello, John. On Wednesday, December 6, 2006, at 3:01:10 PM, you
                              > wrote:
                              >
                              >> At my company, because all test resources are stored in the
                              >> repository, data-dependent tests specify relative paths -- which 'just
                              >> work', assuming a developer has checked out the whole repository and
                              >> not just a subfolder.
                              >
                              >> It's a slick (and becoming standard) way of dealing with the problem
                              >> of external data.
                              >
                              > I must have just not yet correctly guessed the base of the relative
                              > path ...
                              >
                              > Ron Jeffries
                              > www.XProgramming.com
                              > We cannot solve our problems with the same thinking we used when we
                              > created them.
                              > -- Albert Einstein
                              >
                              >
                            • Ron Jeffries
                              Hello, John. On Wednesday, December 6, 2006, at 5:30:33 PM, you ... Cool example, thanks! We probably won t need to go quite this complex, at least not at the
                              Message 14 of 25 , Dec 6, 2006
                              • 0 Attachment
                                Hello, John. On Wednesday, December 6, 2006, at 5:30:33 PM, you
                                wrote:

                                > For this structure, you would reference test data by prefixing it with
                                > the concatenated string ".." + File.separator + ".." + File.separator
                                > + "resources" + File.separator + "testing" + File.separator. Your Ant
                                > targets would take care of specifying the appropriate working
                                > directory for executing the application or running the tests.

                                Cool example, thanks! We probably won't need to go quite this
                                complex, at least not at the start, but a subset will serve as a
                                good skeleton for the future. Thanks again,

                                Ron Jeffries
                                www.XProgramming.com
                                We know less about the project today than at any time in the future.
                                -- Chet Hendrickson
                                You mean today is the dumbest day of the rest of my life?
                                -- Ron Jeffries
                              • Neil Swingler
                                ... Use the classloader to find files relative to the classpath. The Jakarta commons configuration library simplifies this. Then just store the test files in
                                Message 15 of 25 , Dec 6, 2006
                                • 0 Attachment
                                  Ron Jeffries wrote:
                                  > Hello, Donald. On Wednesday, December 6, 2006, at 1:36:06 PM, you
                                  > wrote:
                                  >
                                  >
                                  >> If your test data is in your project, it must be the same on both
                                  >> machines, relatively speaking....
                                  >>
                                  >
                                  > Yes ... so far we haven't figured out quite what the running
                                  > directory is for the tests inside eclipse, and the fact that we
                                  > currently handle our workspaces differently may play in there. I
                                  > suppose I could write a naked file and see where it goes ...
                                  >
                                  >
                                  Use the classloader to find files relative to the classpath. The Jakarta
                                  commons configuration library simplifies this. Then just store the test
                                  files in your java source tree or in a separate directory which you add
                                  to the Eclipse build path as a class folder. Then the test can resolve
                                  the files in a platform and location independent way.

                                  - Neil
                                • Ilja Preuss
                                  ... You can configure that in the launch configuration. I think the default is the projects root directory. Regards, Ilja
                                  Message 16 of 25 , Dec 6, 2006
                                  • 0 Attachment
                                    Ron Jeffries schrieb:
                                    > Yes ... so far we haven't figured out quite what the running
                                    > directory is for the tests inside eclipse

                                    You can configure that in the launch configuration. I think the default
                                    is the projects root directory.

                                    Regards, Ilja
                                  • Donald Roby
                                    ... I think it s the project root normally in Eclipse. You can probably find it by such experimentation or by code. Try this:
                                    Message 17 of 25 , Dec 6, 2006
                                    • 0 Attachment
                                      Ron Jeffries wrote:

                                      >Hello, Donald. On Wednesday, December 6, 2006, at 1:36:06 PM, you
                                      >wrote:
                                      >
                                      >
                                      >
                                      >>If your test data is in your project, it must be the same on both
                                      >>machines, relatively speaking....
                                      >>
                                      >>
                                      >
                                      >Yes ... so far we haven't figured out quite what the running
                                      >directory is for the tests inside eclipse, and the fact that we
                                      >currently handle our workspaces differently may play in there. I
                                      >suppose I could write a naked file and see where it goes ...
                                      >
                                      >
                                      I think it's the project root normally in Eclipse. You can probably
                                      find it by such experimentation or by code.

                                      Try this:

                                      java.util.System.getProperty("user.dir")

                                      >
                                      >
                                      >>You can simply specify a relative path. It may not work if you run your
                                      >>tests outside Eclipse. unless you cd to the project root directory first.
                                      >>
                                      >>
                                      >
                                      >So far, that's not an issue ...
                                      >
                                      >
                                      >
                                      >>Another option is to create a system environment variable for the test
                                      >>directory path, set outside Eclipse so that you can set it differently,
                                      >>and then have a little bit of Java code get that value and append the
                                      >>file name. This might be easier to get working properly under a build
                                      >>script that runs outside your IDE. It's more work, but shouldn't be too
                                      >>bad.
                                      >>Yes ... it seems easy enough ... we just haven't quite scoped out
                                      >>the odd little system calls it takes to do that kind of things.
                                      >>
                                      >>Carl Manaster sent some example code via email that looks like we
                                      >>can make it work. Just a SMOP.
                                      >>
                                      >>
                                      It might be as easy as

                                      java.util.System.getenv("variablename")

                                      but it might depend on the machine and/or the java version. I believe this method was
                                      deprecated and disappeared, but then reappeared, and the behavior may be OS-dependent.
                                    • Ron Jeffries
                                      Thanks to all who are chipping in! ... Hello, Neil. On Wednesday, December 6, 2006, at 6:09:20 PM, you ... Hello, Ilja. On Wednesday, December 6, 2006, at
                                      Message 18 of 25 , Dec 6, 2006
                                      • 0 Attachment
                                        Thanks to all who are chipping in! ...

                                        Hello, Neil. On Wednesday, December 6, 2006, at 6:09:20 PM, you
                                        wrote:

                                        > Use the classloader to find files relative to the classpath. The Jakarta
                                        > commons configuration library simplifies this. Then just store the test
                                        > files in your java source tree or in a separate directory which you add
                                        > to the Eclipse build path as a class folder. Then the test can resolve
                                        > the files in a platform and location independent way.

                                        Hello, Ilja. On Wednesday, December 6, 2006, at 6:50:07 PM, you
                                        wrote:

                                        >> Yes ... so far we haven't figured out quite what the running
                                        >> directory is for the tests inside eclipse

                                        > You can configure that in the launch configuration. I think the default
                                        > is the projects root directory.

                                        Hello, Donald. On Wednesday, December 6, 2006, at 7:16:28 PM, you
                                        wrote:

                                        >>Yes ... so far we haven't figured out quite what the running
                                        >>directory is for the tests inside eclipse, and the fact that we
                                        >>currently handle our workspaces differently may play in there. I
                                        >>suppose I could write a naked file and see where it goes ...
                                        >>
                                        >>
                                        > I think it's the project root normally in Eclipse. You can probably
                                        > find it by such experimentation or by code.

                                        > Try this:

                                        > java.util.System.getProperty("user.dir")
                                        ...
                                        > It might be as easy as
                                        >
                                        > java.util.System.getenv("variablename")
                                        >
                                        > but it might depend on the machine and/or the java version. I believe this method was
                                        > deprecated and disappeared, but then reappeared, and the behavior may be OS-dependent.

                                        Thanks, gang! These ideas should set us on a useful track. As Chet
                                        noted, we'll try something tomorrow when we get together.

                                        Thanks again,

                                        Ron Jeffries
                                        www.XProgramming.com
                                        You do ill if you praise, but worse if you censure,
                                        what you do not understand. --Leonardo da Vinci
                                      • George Dinwiddie
                                        ... It should be the project directory. I got a chuckle out of some of the other suggestions. In the AgileMaryland code project, we ve generally got a windows
                                        Message 19 of 25 , Dec 6, 2006
                                        • 0 Attachment
                                          Ron Jeffries wrote:
                                          > Yes ... so far we haven't figured out quite what the running
                                          > directory is for the tests inside eclipse

                                          It should be the project directory.

                                          I got a chuckle out of some of the other suggestions. In the
                                          AgileMaryland code project, we've generally got a windows and a linux
                                          dev machine going. And we've had OS-X. Mapping drive in batch files
                                          isn't an option.

                                          It does sometimes root out a missed assumption on the part of a
                                          developer. Multi-platform development has its advantages.

                                          - George


                                          --
                                          ----------------------------------------------------------------------
                                          * George Dinwiddie * gdinwiddie@...
                                          Software Development http://www.idiacomputing.com
                                          Consultant and Coach http://www.agilemaryland.org
                                          ----------------------------------------------------------------------
                                        • Ron Jeffries
                                          Hello, George. On Wednesday, December 6, 2006, at 11:27:19 PM, you ... When handed a lemon ... wonder what that person is up to. ;- Thanks, Ron Jeffries
                                          Message 20 of 25 , Dec 7, 2006
                                          • 0 Attachment
                                            Hello, George. On Wednesday, December 6, 2006, at 11:27:19 PM, you
                                            wrote:

                                            > Ron Jeffries wrote:
                                            >> Yes ... so far we haven't figured out quite what the running
                                            >> directory is for the tests inside eclipse

                                            > It should be the project directory.

                                            > I got a chuckle out of some of the other suggestions. In the
                                            > AgileMaryland code project, we've generally got a windows and a linux
                                            > dev machine going. And we've had OS-X. Mapping drive in batch files
                                            > isn't an option.

                                            > It does sometimes root out a missed assumption on the part of a
                                            > developer. Multi-platform development has its advantages.

                                            When handed a lemon ... wonder what that person is up to. ;->

                                            Thanks,

                                            Ron Jeffries
                                            www.XProgramming.com
                                            Testing quality into a program is like spinning straw into gold.
                                            -- George Cameron.
                                          • Kristoffer RoupĂ©
                                            As I write this, I just thought of another way to verify this result. There are 5000-odd pixels in the Hit list for this picture. It would certainly be
                                            Message 21 of 25 , Dec 7, 2006
                                            • 0 Attachment
                                              "As I write this, I just thought of another way to verify this result.
                                              There are 5000-odd pixels in the Hit list for this picture. It would
                                              certainly be possible to spit out the X's and Y's in some format and sum
                                              them, if not by hand (volunteers?) but at least with some other program.
                                              If we got the same result, we'd be even more sure than we are now that we
                                              have the right answer. Maybe we'll do that. In any case, the test above is
                                              a "Golden Master" test, in that we checked the result somehow, and now
                                              accept it as the truth."

                                              First thing that popped my mind was to generate (paint?) a file for this.
                                              Put a couple of circles inside eachother in a picture, that should be
                                              fairly easy to check with a human/customer eye.

                                              //Kristoffer
                                            • martijn_meijering
                                              ... I agree with the suggestions made by you and others to keep the data files, tests etc in the repository itself. That s what I usually do. You make an
                                              Message 22 of 25 , Dec 7, 2006
                                              • 0 Attachment
                                                > > Are you keeping your data files outside your project directory and
                                                > > version control system?
                                                >
                                                > They're currently at the top of the project hierarchy, if I recall.
                                                > I don't know whether they are checked in or not, but certainly they
                                                > should be.
                                                >
                                                > We're really at the bottom of the food chain on this, we don't
                                                > really know what the standard ways of doing this are. We're sure
                                                > they're out there, just haven't googled them out.
                                                >
                                                > That's why I asked our remote pairs. How would you do it?

                                                I agree with the suggestions made by you and others to keep the data
                                                files, tests etc in the repository itself. That's what I usually do.

                                                You make an interesting point about golden master tests in your
                                                article that I'd like to draw people's attention to. As you say, they
                                                are great for checking stuff stays the same, but not so great for
                                                checking it is correct. I have really positive experiences with the
                                                former and have been bitten by the latter.

                                                If you are faced with a mountain of legacy code without tests, writing
                                                simple system tests that generate output files of some sort is a great
                                                way to get started. It is very easy to do, and with the help of a code
                                                coverage tool and a domain expert it is easy enough to get decent
                                                coverage. Plus, if you compare output files byte for byte, you get
                                                very sensitive tests. The combination of low cost, good coverage and
                                                high sensitivity gives you a workable safety net to use when refactoring.

                                                It is not nearly as great as having unit tests (system tests are
                                                likely to run more slowly, don't tell you which module probably
                                                contains the bug and so on), but I've found it to be a good start.
                                                Michael Feather's book is full of useful tips on how to get started
                                                writing unit tests for legacy code. I'd still recommend starting with
                                                the regression tests, but I wish I had read his book before I started
                                                work on a recent restructuring project. Next time around I'd use both
                                                methods.

                                                The high sensitivity is of course also a drawback when you start
                                                changing functionality, instead of just refactoring, especially with
                                                numerical code, where small variations are not necessarily wrong.

                                                All in all I see golden master tests as a very good point to start in
                                                some situations, but I'd like to move away from them and start writing
                                                proper unit and functional tests as the project progresses. One area
                                                where I see a long term role for them is for comparing the output of
                                                simulation software with experimental data or the output of other
                                                simulators. There I'd want to use some kind of function space norm to
                                                compare solutions instead of a test for equality.

                                                Just a few comments triggered by your article...
                                              • Ron Jeffries
                                                Hello, Kristoffer. On Thursday, December 7, 2006, at 9:33:13 AM, ... We do plan to do that, to show the customer what the center looks like. I m more
                                                Message 23 of 25 , Dec 7, 2006
                                                • 0 Attachment
                                                  Hello, Kristoffer. On Thursday, December 7, 2006, at 9:33:13 AM,
                                                  you wrote:

                                                  > "As I write this, I just thought of another way to verify this result.
                                                  > There are 5000-odd pixels in the Hit list for this picture. It would
                                                  > certainly be possible to spit out the X's and Y's in some format and sum
                                                  > them, if not by hand (volunteers?) but at least with some other program.
                                                  > If we got the same result, we'd be even more sure than we are now that we
                                                  > have the right answer. Maybe we'll do that. In any case, the test above is
                                                  > a "Golden Master" test, in that we checked the result somehow, and now
                                                  > accept it as the truth."

                                                  > First thing that popped my mind was to generate (paint?) a file for this.
                                                  > Put a couple of circles inside eachother in a picture, that should be
                                                  > fairly easy to check with a human/customer eye.

                                                  We do plan to do that, to show the customer what the center looks
                                                  like. I'm more concerned in this case with the (I claim slim)
                                                  possibility that our calculation is off in some way that might
                                                  matter but not be easily visible. I like tests to be hard and fast,
                                                  not fuzzy ... even when, as in this case, the point of the product
                                                  is to help the user visualize what's going on, not to provide tight
                                                  statistical correctness.

                                                  Ron Jeffries
                                                  www.XProgramming.com
                                                  How do I know what I think until I hear what I say? -- E M Forster
                                                • ken_shrum@agilent.com
                                                  ... If you had written this test first, as opposed to knowing what the answer is (the average of x and the average of y), what would your tests have looked
                                                  Message 24 of 25 , Dec 7, 2006
                                                  • 0 Attachment
                                                    > We do plan to do that, to show the customer what the center looks
                                                    > like. I'm more concerned in this case with the (I claim slim)
                                                    > possibility that our calculation is off in some way that might
                                                    > matter but not be easily visible. I like tests to be hard and fast,
                                                    > not fuzzy ... even when, as in this case, the point of the product
                                                    > is to help the user visualize what's going on, not to provide tight
                                                    > statistical correctness.

                                                    If you had written this test first, as opposed to knowing what
                                                    the answer is (the average of x and the average of y), what would
                                                    your tests have looked like? I think you got sucked into a
                                                    "let's use real data" mindset.

                                                    If it had been me, for example, I would have started with
                                                    a trivial gif with a small round dot, then perhaps a "full" gif
                                                    (completely solid, so the midpoint is the middle of the gif),
                                                    and then finally a line at some arbitrary angle, so the middle
                                                    is the midpoint of the line.

                                                    Ken
                                                  • Ron Jeffries
                                                    Hello, ken. On Thursday, December 7, 2006, at 1:49:02 PM, you ... We do have a little test, in fact three of them ... ... Ours are different from that but
                                                    Message 25 of 25 , Dec 7, 2006
                                                    • 0 Attachment
                                                      Hello, ken. On Thursday, December 7, 2006, at 1:49:02 PM, you
                                                      wrote:

                                                      >> We do plan to do that, to show the customer what the center looks
                                                      >> like. I'm more concerned in this case with the (I claim slim)
                                                      >> possibility that our calculation is off in some way that might
                                                      >> matter but not be easily visible. I like tests to be hard and fast,
                                                      >> not fuzzy ... even when, as in this case, the point of the product
                                                      >> is to help the user visualize what's going on, not to provide tight
                                                      >> statistical correctness.

                                                      > If you had written this test first, as opposed to knowing what
                                                      > the answer is (the average of x and the average of y), what would
                                                      > your tests have looked like? I think you got sucked into a
                                                      > "let's use real data" mindset.

                                                      We do have a little test, in fact three of them ...

                                                      > If it had been me, for example, I would have started with
                                                      > a trivial gif with a small round dot, then perhaps a "full" gif
                                                      > (completely solid, so the midpoint is the middle of the gif),
                                                      > and then finally a line at some arbitrary angle, so the middle
                                                      > is the midpoint of the line.

                                                      Ours are different from that but similar in concept ...

                                                      Still, looking at the big file, while I'm "confident" that the
                                                      center is correct, I don't know that it is with the same confidence
                                                      that I have for my three little tests.

                                                      Ron Jeffries
                                                      www.XProgramming.com
                                                      Do as you will, try to do it well. That's what I do.
                                                    Your message has been successfully submitted and would be delivered to recipients shortly.