Loading ...
Sorry, an error occurred while loading the content.
 

Re: [extremeperl] Testing, Audience, etc.

Expand Messages
  • Tom Brown
    ... but it s not. An array in a scalar context is the number of elements in it. I don t need to tell you that, your knowledge of the language is better than
    Message 1 of 20 , Jan 29, 2002
      > > >
      > > > $second_arg = 35 if int(@_) < 3;
      > >
      > >
      > > So, you only use it, because it is more descriptive?
      >
      > I guess so. perhaps all of the examples I have seen used int() on
      > arrays. It seems intuitive to me as well.

      but it's not. An array in a scalar context is the number of elements in
      it. I don't need to tell you that, your knowledge of the language is
      better than mine....

      adding the int() function makes me look at _why_ you are adding the
      function, and try to find the twist that needs it. There isn't one, so
      I've wasted time figuring that out. That is why I prefer either scalar()
      or nothing.

      Obviously, it's mostly a matter of style. Does it matter? I can't answer
      that. For the book to be used/popular you probably want to try to code to
      your target audience. No individual can claim to represent that.

      -Tom
    • Rob Nagler
      ... I m not so sure about that. ... Nothing like data to backup a hypothesis: trgrep bint (? s*@ /usr/lib/perl | grep -v Bivio | wc 6 trgrep
      Message 2 of 20 , Jan 29, 2002
        Tom Brown writes:
        > but it's not. An array in a scalar context is the number of elements in
        > it. I don't need to tell you that, your knowledge of the language is
        > better than mine....

        I'm not so sure about that.

        > adding the int() function makes me look at _why_ you are adding the
        > function, and try to find the twist that needs it. There isn't one, so
        > I've wasted time figuring that out. That is why I prefer either scalar()
        > or nothing.

        Nothing like data to backup a hypothesis:

        trgrep '\bint\(?\s*@' /usr/lib/perl | grep -v Bivio | wc
        6

        trgrep '\bscalar\(?\s*@' /usr/lib/perl | grep -v Bivio | wc
        157

        The scalars have it. I'll change it to be nothing or scalar where
        required, thanks.

        > Obviously, it's mostly a matter of style. Does it matter? I can't answer
        > that. For the book to be used/popular you probably want to try to code to
        > your target audience. No individual can claim to represent that.

        This is the danger of writing a book about perl and XP. Whenever I
        mention that I program in perl at an XP meeting, I get the strangest
        looks. Fortunately, the perl community is more forgiving about XP.
        Yet they aren't the easiest audience to please either. :)

        Cheers,
        Rob
      • chromatic
        ... I ve had fairly good results combining the two. It s convincing free software folks to write tests that s tricky. -- chromatic
        Message 3 of 20 , Jan 29, 2002
          On Tuesday 29 January 2002 13:15, you wrote:

          > This is the danger of writing a book about perl and XP. Whenever I
          > mention that I program in perl at an XP meeting, I get the strangest
          > looks. Fortunately, the perl community is more forgiving about XP.
          > Yet they aren't the easiest audience to please either. :)

          I've had fairly good results combining the two. It's convincing free
          software folks to write tests that's tricky.

          -- chromatic
        • Ged Haywood
          Hi Rob, ... I spent the better part of the previous two years doing both, for a system with something approaching 4 million users world-wide (IE and Netscape,
          Message 4 of 20 , Jan 29, 2002
            Hi Rob,

            On Tue, 29 Jan 2002, Rob Nagler wrote:

            > I am not a GUI programmer, and I have very little experience testing GUIs.

            I spent the better part of the previous two years doing both, for a
            system with something approaching 4 million users world-wide (IE and
            Netscape, versions 4+ only). If there's anything you want to ask, do
            please ask. I won't know the answer.

            73,
            Ged.
          • Ed Grimm
            ... Correction: it s convincing *anybody* to write tests when they haven t learned to on their own that s tricky. I ve also noticed it s rather difficult to
            Message 5 of 20 , Feb 1, 2002
              On Tue, 29 Jan 2002, chromatic wrote:
              > On Tuesday 29 January 2002 13:15, you wrote:
              >
              >> This is the danger of writing a book about perl and XP. Whenever I
              >> mention that I program in perl at an XP meeting, I get the strangest
              >> looks. Fortunately, the perl community is more forgiving about XP.
              >> Yet they aren't the easiest audience to please either. :)
              >
              > I've had fairly good results combining the two. It's convincing free
              > software folks to write tests that's tricky.

              Correction: it's convincing *anybody* to write tests when they haven't
              learned to on their own that's tricky.

              I've also noticed it's rather difficult to get a lot of XPers to
              understand you can do it outside their chosen language. Except that I
              have seen some Java XPers that seemed to understand you can do it in
              SmallTalk.

              Ed
            • drewbie74
              Well, as a new convert to writing tests I can attest to your hypothesis. In general I have not sat down and written a good test suite when coding something
              Message 6 of 20 , Feb 5, 2002
                Well, as a new convert to writing tests I can attest to your
                hypothesis. In general I have not sat down and written a good test
                suite when coding something new. But I did so with a new module I
                just wrote and I'm sold on the utility of tests.

                Case in point. I wanted to make getting config vars case insensitive.
                So I saved everything as lowercase hash keys. But this broke
                something else, so I had to go back. Because I had tests, it was very
                easy to see when everything was working correctly again. This benefit
                alone is what really sold me on tests.

                How to convince people to wite tests? Chromatic's article on perl.com
                was excellent in showing HOW to write tests. Perhaps we need an short
                article showing exactly how tests made a project easier? Heck, I've
                got time, maybe I'll put something together...

                Drew

                --- In extremeperl@y..., chromatic <chromatic@r...> wrote:

                > I've had fairly good results combining the two. It's convincing
                free
                > software folks to write tests that's tricky.
                >
                > -- chromatic
              • chromatic
                ... How about some brainstorming then? Here are several benefits of writing tests, in no particular order: - to clarify the intent of the code - to enforce
                Message 7 of 20 , Feb 6, 2002
                  On Tuesday 05 February 2002 16:50, drewbie74 wrote:

                  > How to convince people to wite tests? Chromatic's article on perl.com
                  > was excellent in showing HOW to write tests. Perhaps we need an short
                  > article showing exactly how tests made a project easier? Heck, I've
                  > got time, maybe I'll put something together...

                  How about some brainstorming then? Here are several benefits of writing
                  tests, in no particular order:

                  - to clarify the intent of the code
                  - to enforce behavioral compatibility
                  - to explore boundary conditions
                  - to ensure that bugs have been corrected
                  - to ensure that bugs remain corrected
                  - to exercise an interface
                  - to help with decoupling (when writing code for testability)
                  - to add explanations of behavior (when using good test names with
                  Test::More)

                  There's obviously more (to make refactoring possible), but there's a
                  disturbing tendency to rewrite software from scratch, and I'm picking my
                  battles for now.

                  -- c
                • Drew Taylor
                  ... Well, based on some work I did last night I have a prime example. I had written some modules previously that worked and are currently in production. Since
                  Message 8 of 20 , Feb 6, 2002
                    At 11:49 AM 2/6/2002 -0800, chromatic wrote:

                    >On Tuesday 05 February 2002 16:50, drewbie74 wrote:
                    >
                    > > How to convince people to wite tests? Chromatic's article on perl.com
                    > > was excellent in showing HOW to write tests. Perhaps we need an short
                    > > article showing exactly how tests made a project easier? Heck, I've
                    > > got time, maybe I'll put something together...
                    >
                    >How about some brainstorming then? Here are several benefits of writing
                    >tests, in no particular order:
                    >
                    > - to clarify the intent of the code
                    > - to enforce behavioral compatibility
                    > - to explore boundary conditions
                    > - to ensure that bugs have been corrected
                    > - to ensure that bugs remain corrected
                    > - to exercise an interface
                    > - to help with decoupling (when writing code for testability)
                    > - to add explanations of behavior (when using good test names with
                    >Test::More)
                    >
                    >There's obviously more (to make refactoring possible), but there's a
                    >disturbing tendency to rewrite software from scratch, and I'm picking my
                    >battles for now.
                    >
                    >-- c

                    Well, based on some work I did last night I have a prime example. I had
                    written some modules previously that worked and are currently in
                    production. Since there were no tests for said modules, I begun witing them
                    last night using Test::More (Thank you SO much Schwern - it rocks!). In the
                    process of doing so, I ended up doing some refactoring to make the modules
                    more flexible and worked out a couple of possible bugs. In the end, I had
                    better, more flexible, and refactored code that had good,comprehensive
                    tests - fulfilling the last 5 items in your list above. I was also forced
                    to setup a development environment (including a db) that had test data and
                    which I could use for testing purposes in perpetuity. This alone was worth
                    it's weight in gold.

                    For me, the acts of refactoring and writing tests now go hand in hand. Yes,
                    it takes time, sometimes a lot. But it is time well spent because it makes
                    your code better by the very nature of writing tests & making sure they
                    correctly run. It's hard to do one without the other because it's difficult
                    to make sure your "enhancement" isn't really a bug.

                    To address your last point, having tests helps keep me from rewriting code.
                    The tests help keep me from rewriting because I can fix only what is broken
                    or needs enhancing rather than thinking "This code is a bunch of crap!
                    Let's throw it out and just start over." Starting over is likely to
                    introduce more bugs than you fix. And this of course is exactly what you
                    were trying to avoid in the first place.

                    Oh geez, I hope I'm not becoming a zealot. Not that it would be a bad thing...


                    Well, based on some work I did last night I have a prime example. I had
                    written some modules previously that worked and are currently in
                    production. Since there were no tests for said modules, I begun witing them
                    last night using Test::More (Thank you SO much Schwern - it rocks!). In the
                    process of doing so, I ended up doing some refactoring to make the modules
                    more flexible and worked out a couple of possible bugs. In the end, I had
                    better, more flexible, and refactored code that had good,comprehensive
                    tests - fulfilling the last 5 items in your list above. I was also forced
                    to setup a development environment (including a db) that had test data and
                    which I could use for testing purposes in perpetuity. This alone was worth
                    it's weight in gold.

                    For me, the acts of refactoring and writing tests now go hand in hand. Yes,
                    it takes time, sometimes a lot. But it is time well spent because it makes
                    your code better by the very nature of writing tests & making sure they
                    correctly run. It's hard to do one without the other because it's difficult
                    to make sure your "enhancement" isn't really a bug.

                    To address your last point, having tests helps keep me from rewriting code.
                    The tests help keep me from rewriting because I can fix only what is broken
                    or needs enhancing rather than thinking "This code is a bunch of crap!
                    Let's throw it out and just start over." Starting over is likely to
                    introduce more bugs than you fix. And this of course is exactly what you
                    were trying to avoid in the first place.

                    Oh geez, I hope I'm not becoming a zealot. Not that it would be a bad thing...

                    Drew Taylor JA[P|m_p|SQL]H
                    http://www.drewtaylor.com/ Just Another Perl|mod_perl|SQL Hacker
                    mailto:drew@... *** God bless America! ***
                  • Rob Nagler
                    ... I like to distinguish between acceptance and unit tests. For unit tests, I would add: - to validate the API - to enable refactoring - to keep the cost of
                    Message 9 of 20 , Feb 6, 2002
                      Drew Taylor writes:
                      > At 11:49 AM 2/6/2002 -0800, chromatic wrote:
                      > >How about some brainstorming then? Here are several benefits of writing
                      > >tests, in no particular order:

                      I like to distinguish between acceptance and unit tests. For unit
                      tests, I would add:

                      - to validate the API
                      - to enable refactoring
                      - to keep the cost of change constant

                      A unit test suite is like a semantic compiler. In dynamic languages,
                      like Perl, there needs to be something to validate the code.

                      For acceptance tests, I would add:

                      - to encode concisely the domain knowledge of the customer
                      - to help the development team understand the problem

                      > Oh geez, I hope I'm not becoming a zealot. Not that it would be a bad thing...

                      Praise the Tests brother!

                      Rob
                    • Drew Taylor
                      Could you explain a little more about the two types of tests you mention below? Specifically the acceptance tests, and how they would relate to web apps. I m
                      Message 10 of 20 , Feb 6, 2002
                        Could you explain a little more about the two types of tests you mention
                        below? Specifically the acceptance tests, and how they would relate to web
                        apps. I'm trying to become more of an "architect", so a lot of the "domain
                        knowledge" stuff you & other texts mention is still a little fuzzy to me. I
                        read some of the docs on bivio, and it sounded very interesting.
                        Unfortunately more of it than I wanted was over my head. I'm very intrigued
                        about the platform you released and how to learn more about it in the
                        future - it would be a great learning tool for me.

                        It sounds like the tests I wrote last night would be unit tests as they
                        test object instantiation & the APIs.

                        Drew

                        At 03:35 PM 2/6/2002 -0700, Rob Nagler wrote:

                        >I like to distinguish between acceptance and unit tests. For unit
                        >tests, I would add:
                        >
                        >- to validate the API
                        >- to enable refactoring
                        >- to keep the cost of change constant
                        >
                        >A unit test suite is like a semantic compiler. In dynamic languages,
                        >like Perl, there needs to be something to validate the code.
                        >
                        >For acceptance tests, I would add:
                        >
                        >- to encode concisely the domain knowledge of the customer
                        >- to help the development team understand the problem

                        Drew Taylor JA[P|m_p|SQL]H
                        http://www.drewtaylor.com/ Just Another Perl|mod_perl|SQL Hacker
                        mailto:drew@... *** God bless America! ***
                      • Rob Nagler
                        ... There are many different kinds of tests including performance and load testing. In XP we re mostly concerned about unit and acceptance testing. ... What
                        Message 11 of 20 , Feb 6, 2002
                          Drew Taylor writes:
                          > Could you explain a little more about the two types of tests you mention
                          > below?

                          There are many different kinds of tests including performance and load
                          testing. In XP we're mostly concerned about unit and acceptance
                          testing.

                          > Specifically the acceptance tests, and how they would relate to web
                          > apps.

                          What is beautiful about the Web is that HTTP and HTML are a messaging
                          interface. You can build a complete acceptance test suite without
                          dealing with GUI scripting.

                          An acceptance test is a way of verifying end-user functions. A unit
                          test verifies programmer level functions. Both can test Web software.

                          > I'm trying to become more of an "architect",

                          Before you do, read this article:

                          http://joel.editthispage.com/stories/storyReader$320

                          One of the things I'm trying to learn is to become less of an architect.

                          > so a lot of the "domain
                          > knowledge" stuff you & other texts mention is still a little fuzzy
                          > to me.

                          Domain knowledge is simply "the problem". What I like about XP is
                          that it is problem-oriented, not solution-oriented.

                          > I
                          > read some of the docs on bivio, and it sounded very interesting.
                          > Unfortunately more of it than I wanted was over my head. I'm very intrigued
                          > about the platform you released and how to learn more about it in the
                          > future - it would be a great learning tool for me.

                          The great thing about perl is that lots of people have created lots of
                          code. We put out bOP, because it has no intrinsic value as a
                          product. There are just too many good toolkits out there. I consider
                          this a testament to perl more than anything else. You can create
                          incredibly solid software very quickly.

                          > It sounds like the tests I wrote last night would be unit tests as they
                          > test object instantiation & the APIs.

                          Yes, it sounds like you wrote unit tests. They are incredibly
                          important tools. We're slowly creating unit tests for our code. It's
                          tough to do, but we regret it every time we make changes and there is
                          no test to validate that we haven't broken anything.

                          Hope this helps.

                          Rob
                        • Drew Taylor
                          ... How do you do that? I ve only heard of tools that allow you to create a script and then playback that script. This is semiuseful, but what happens if you
                          Message 12 of 20 , Feb 6, 2002
                            At 10:52 PM 2/6/2002 -0700, Rob Nagler wrote:
                            >Drew Taylor writes:
                            > > Specifically the acceptance tests, and how they would relate to web
                            > > apps.
                            >
                            >What is beautiful about the Web is that HTTP and HTML are a messaging
                            >interface. You can build a complete acceptance test suite without
                            >dealing with GUI scripting.
                            >An acceptance test is a way of verifying end-user functions. A unit
                            >test verifies programmer level functions. Both can test Web software.

                            How do you do that? I've only heard of tools that allow you to create a
                            script and then playback that script. This is semiuseful, but what happens
                            if you make a change to the interface? What tools/techniques have you used
                            in the past to do acceptance testing? Do I have to setup a fake web server
                            environment & run the tests that way? That wouldn't be too difficult in a
                            CGI environment, and there are things like Apache::Fake now.

                            > > I'm trying to become more of an "architect",
                            >
                            >Before you do, read this article:
                            >
                            >http://joel.editthispage.com/stories/storyReader$320
                            >
                            >One of the things I'm trying to learn is to become less of an architect.

                            I hadn't seen that one before, although I have read some of Joel's other
                            articles. I'll read it tomorrow morning. My brief look says it will be
                            good. What I'm ultimately interested in learning is better design. Learning
                            patterns is one step, and working to see the problem from a higher level
                            view are two things I'm doing now.

                            > > so a lot of the "domain
                            > > knowledge" stuff you & other texts mention is still a little fuzzy
                            > > to me.
                            >
                            >Domain knowledge is simply "the problem". What I like about XP is
                            >that it is problem-oriented, not solution-oriented.

                            OK. Why don't they just say that? :-) I know that at high levels it's
                            essential that we're all speaking the same language, but can't that
                            language be simpler?

                            > > I
                            > > read some of the docs on bivio, and it sounded very interesting.
                            > > Unfortunately more of it than I wanted was over my head. I'm very
                            > intrigued
                            > > about the platform you released and how to learn more about it in the
                            > > future - it would be a great learning tool for me.
                            >
                            >The great thing about perl is that lots of people have created lots of
                            >code. We put out bOP, because it has no intrinsic value as a
                            >product. There are just too many good toolkits out there. I consider
                            >this a testament to perl more than anything else. You can create
                            >incredibly solid software very quickly.

                            This ability to quickly create a great product is one of the things that
                            really attracted me once I got serious about perl. I've seen several
                            frameworks that have interested me, including OpenInteract, OpenFrame, and
                            Mason. One day I hope I have the time to put some effort into learning each
                            better. I can just imagine all the tidbits of knowledge waiting to be
                            gleaned from each one.

                            > > It sounds like the tests I wrote last night would be unit tests as they
                            > > test object instantiation & the APIs.
                            >
                            >Yes, it sounds like you wrote unit tests. They are incredibly
                            >important tools. We're slowly creating unit tests for our code. It's
                            >tough to do, but we regret it every time we make changes and there is
                            >no test to validate that we haven't broken anything.

                            Yep, they were definitely unit tests. And as I mentioned before, I'm very
                            grateful I've written the ones I have because they did exactly what they're
                            supposed to do. Tell me when bugs appear and when they've been fixed. Once
                            I've gotten more tests done, I need to look into Test::Harness so I can run
                            them all at one swoop.

                            To illuistrate your last point, I have an example. At a previous employer,
                            we had a large codebase of perl modules (I bet it's probably doubled by
                            now) but no tests. I'm still close friends w/ the lead QA person and she
                            often just tests what she can and blindly hopes everything else still
                            works. It's just not possible to test every facet of the code for every
                            release (which is every 1-2 months). I really doubt that a comprehensive
                            test suite will ever be written, even though I have no doubt that it would
                            be an extremely important tool. The CTO's not convinced of the need, and I
                            don't think they would have/put the time to write a comprehensive suite
                            anyway. Besides, they're probably a little afraid of all the little bugs it
                            might turn up. ;-)

                            Could this be a hidden reason some shops are wary of tests?

                            Drew
                            Drew Taylor JA[P|m_p|SQL]H
                            http://www.drewtaylor.com/ Just Another Perl|mod_perl|SQL Hacker
                            mailto:drew@... *** God bless America! ***
                          • chromatic
                            ... If it s hidden, it s not hidden very well. :) Here s my take. Programmers don t want to write tests because: - it s not their job - it s not real
                            Message 13 of 20 , Feb 6, 2002
                              On Wednesday 06 February 2002 23:33, Drew Taylor wrote:

                              > The CTO's not convinced of the need, and I don't think they would have/put
                              > the time to write a comprehensive suite anyway. Besides, they're probably a
                              > little afraid of all the little bugs it might turn up. ;-)

                              > Could this be a hidden reason some shops are wary of tests?

                              If it's hidden, it's not hidden very well. :) Here's my take.

                              Programmers don't want to write tests because:

                              - it's not their job
                              - it's not "real" coding
                              - it's not as sexy as "real" coding
                              - they don't know how
                              - they don't know (or believe) the benefits
                              - it's hard (but only the smart ones really believe this)

                              Managers don't want coders to write tests because:

                              - it takes time away from "real" coding
                              - QA should handle it
                              - it's cheaper to fix bugs when they're found

                              I'm only sympathetic to the coders who don't yet know how and those who think
                              writing tests is difficult. (I'll even propose that, unless you're adding
                              tests to a system that has none, it *shouldn't* be difficult. If it is,
                              you're not coding for testability and you're asking for trouble.)

                              Okay, my analyst hat is off. Feel free to jump on this thread with
                              evangelism if anyone has questions.

                              * * * * *

                              As for your Test::Harness question, use h2xs to make a skeleton Makefile.PL
                              for your project. Put your tests in the t/ subdirectory, edit the @INC paths
                              if needed, and run 'perl Makefile.PL; make; make test' and it should Just
                              Work.

                              Now you know just about as much as I care to remember about the whole process.

                              -- c
                            • Drew Taylor
                              ... All your points are right on the money. I would venture to guess that most good programmers would not be against writing tests. IMHO, it s usually
                              Message 14 of 20 , Feb 7, 2002
                                At 11:45 PM 2/6/2002 -0700, chromatic wrote:

                                ><snip> excellent points </snip>

                                >I'm only sympathetic to the coders who don't yet know how and those who think
                                >writing tests is difficult. (I'll even propose that, unless you're adding
                                >tests to a system that has none, it *shouldn't* be difficult. If it is,
                                >you're not coding for testability and you're asking for trouble.)

                                All your points are right on the money. I would venture to guess that most
                                "good" programmers would not be against writing tests. IMHO, it's usually
                                managements edicts & timelines that forces the lack of tests. As for the
                                last point, I read somewhere (perhaps in one of Steve McConnell's books)
                                that a study found it takes magnitudes of more time/money to fix a bug
                                after the fact that to fix it before. And if it negatively affects a
                                customer, then multiply that cost by several factors for pissed off said
                                customer. If only management could be made to understand this fact, getting
                                tests would be much simpler. If I ever get a fulltime job again, I
                                certainly will push for good tests, even if it's just unit tests.

                                I was once one of those people who didn't understand how easy it is to
                                write tests. Yes, it took some time to get a good test environment setup,
                                but it was worth the few hous it took. And now I can easily and _quickly_
                                add new tests. Modules like Test::Simple & Test::More are the key.

                                >As for your Test::Harness question, use h2xs to make a skeleton Makefile.PL
                                >for your project. Put your tests in the t/ subdirectory, edit the @INC paths
                                >if needed, and run 'perl Makefile.PL; make; make test' and it should Just
                                >Work.

                                That's almost too easy... But then, this IS perl we're talking about. :-)

                                [OT] As an aside, why is it that when I do Reply TO All in Eudora that the
                                mailing list is on the to line twice, rather than the poster & the list?
                                It's rather annoying since that is my usual behavior. Or is this considered
                                bad form? I've never gotten a good answer on this question.

                                Drew
                                Drew Taylor JA[P|m_p|SQL]H
                                http://www.drewtaylor.com/ Just Another Perl|mod_perl|SQL Hacker
                                mailto:drew@... *** God bless America! ***
                              • Ged Haywood
                                Hi all, ... [snip] ... See attached. I once had to fly to seven or eight different countries over a period of several weeks to fix a $0.25 problem in a couple
                                Message 15 of 20 , Feb 7, 2002
                                  Hi all,

                                  On Thu, 7 Feb 2002, Drew Taylor wrote:

                                  > At 11:45 PM 2/6/2002 -0700, chromatic wrote:
                                  [snip]
                                  > managements edicts & timelines that forces the lack of tests. As for the
                                  > last point, I read somewhere (perhaps in one of Steve McConnell's books)
                                  > that a study found it takes magnitudes of more time/money to fix a bug
                                  > after the fact that to fix it before. And if it negatively affects a
                                  > customer, then multiply that cost by several factors for pissed off said
                                  > customer. If only management could be made to understand this fact...

                                  See attached.

                                  I once had to fly to seven or eight different countries over a period
                                  of several weeks to fix a $0.25 problem in a couple of hundred $22,000
                                  instruments because the guy in procurement had ignored my written
                                  procurement specification and the guy in test had ignored my written
                                  test secification. The problems only started to surface when the
                                  instruments were used in hot places.

                                  It still bugs me that I didn't send a bill to their employer, who was
                                  my supplier and contracted to make the things to the specification.

                                  It's called experience.

                                  73,
                                  Ged.


                                  [Non-text portions of this message have been removed]
                                • Rob Nagler
                                  ... The first step is to make sure your interface has structure. If you are testing arbitrarily constructed templates, you re going to have a rough time. If
                                  Message 16 of 20 , Feb 8, 2002
                                    > How do you do that? I've only heard of tools that allow you to create a
                                    > script and then playback that script. This is semiuseful, but what happens
                                    > if you make a change to the interface?

                                    The first step is to make sure your interface has structure. If you
                                    are testing arbitrarily constructed templates, you're going to have a
                                    rough time. If you build your HTML pages from widgets or
                                    parameterized templates, you'll have some structure to grab on to.

                                    > What tools/techniques have you used
                                    > in the past to do acceptance testing?

                                    The best tool is perl. We used it to test our CORBA based Web server,
                                    and we use it to test our application written in perl. It is not very
                                    hard to build an acceptance test suite using tools like LWP and
                                    HTMLParser.

                                    I am a little behind schedule. My goal is to release our internal
                                    infrastructure by the end of the month. It will come with a test
                                    suite which tests our pet shop demo (http://petshop.bivio.biz).

                                    > Do I have to setup a fake web server
                                    > environment & run the tests that way?

                                    I find for acceptance testing you need a test environment which is as
                                    close to your production environment as possible. Any of our
                                    developers can run the test suite on their personal Web servers.
                                    Every night we run the test suite against our test servers, which are
                                    relatively clean machines.

                                    > good. What I'm ultimately interested in learning is better design. Learning
                                    > patterns is one step, and working to see the problem from a higher level
                                    > view are two things I'm doing now.

                                    To me, there are two sides: analysis and synthesis. Patterns are
                                    about synthesis. XP is about analysis. I'm not a big fan of
                                    patterns, because they're very focused on classical object-oriented
                                    programming, and I try to program declaratively whenever I can. In
                                    addition, languages like Java, have some serious deficiencies such as
                                    weak ability to delegate and no class level inheritance.

                                    I find reading books about Lisp, functional programming, and logic
                                    programming expands my solution set much better than reading a book
                                    about design patterns.

                                    > OK. Why don't they just say that? :-) I know that at high levels it's
                                    > essential that we're all speaking the same language, but can't that
                                    > language be simpler?

                                    It is really hard to write cogent prose, which addresses a wide
                                    audience. They're just some concepts which are hard to explain in
                                    simpler language. I'm reading a book by Einstein which is incredibly
                                    well written but I have a really hard time understanding his
                                    discussions about the special theory and general theory of
                                    relativity. That's why I'm a programmer, I guess.

                                    > anyway. Besides, they're probably a little afraid of all the little bugs it
                                    > might turn up. ;-)
                                    >
                                    > Could this be a hidden reason some shops are wary of tests?

                                    I don't think so. In general, people have a hard time quantifying
                                    quality. If it works in the general case, it may be enough. The user
                                    base may be small. I really like Gerry Weinberg's comments on quality
                                    in his book "Quality Software Management: Vol. 1 Systems Thinking":

                                    The Quality Statement: Every statement about quality is a statement about some
                                    person(s).

                                    The Political Dilemma: More quality for one person may mean less
                                    quality for another.

                                    The Quality Decision: Whose opinion of quality is to count when making
                                    decisions?

                                    The Inadequate Definition of Quality: Quality is the absence of error.

                                    The Absence of Errors Fallacy: Though copious errors guarantees
                                    worthlessness, having zero errors guarantees nothing at all about the
                                    value of software.

                                    I highly recommend the book.

                                    Rob
                                  • Stas Bekman
                                    ... Since you are talking about testing apps against webserver, I d plug in the new Apache::Test framework which most Apache::* modules and frameworks
                                    Message 17 of 20 , Feb 21, 2002
                                      Rob Nagler wrote:

                                      >>What tools/techniques have you used
                                      >>in the past to do acceptance testing?
                                      >>
                                      >
                                      > The best tool is perl. We used it to test our CORBA based Web server,
                                      > and we use it to test our application written in perl. It is not very
                                      > hard to build an acceptance test suite using tools like LWP and
                                      > HTMLParser.
                                      >
                                      > I am a little behind schedule. My goal is to release our internal
                                      > infrastructure by the end of the month. It will come with a test
                                      > suite which tests our pet shop demo (http://petshop.bivio.biz).
                                      >
                                      >
                                      >>Do I have to setup a fake web server
                                      >>environment & run the tests that way?
                                      >>
                                      >
                                      > I find for acceptance testing you need a test environment which is as
                                      > close to your production environment as possible. Any of our
                                      > developers can run the test suite on their personal Web servers.
                                      > Every night we run the test suite against our test servers, which are
                                      > relatively clean machines.

                                      Since you are talking about testing apps against webserver, I'd plug in
                                      the new Apache::Test framework which most Apache::* modules' and
                                      frameworks' developers will find very helpful. The goal is to have every
                                      Apache::* module needing mod_perl or just plain apache env, use
                                      Apache::Test for its test. There is no more excuses for not having
                                      tests. And if something is missing from its functionality now it's the
                                      time to jump in and ask for it/add it.

                                      httpd-test project is using this Perl framework for testing C modules
                                      for Apache 1.3 and 2.0 and the server itself. And of course originally
                                      it was developed for mod_perl 2.0. The same test suite can work with
                                      httpd 1.3 and httpd 2.0. For more info see:
                                      http://perl.apache.org/preview/modperl-site/docs/2.0/devel/testing/testing.html

                                      Once we release the new modperl site (hopefully in a few weeks) this URL
                                      will appear as:
                                      http://perl.apache.org/docs/2.0/devel/testing/testing.html

                                      I've started mentioning XP in this doc and mention reasons for a need to
                                      test, but more work in needed so your help is very welcome.

                                      To get the framework grab the httpd-test rep from cvs or the snapshot:
                                      http://cvs.apache.org/snapshots/modperl-2.0/

                                      also see:
                                      http://cvs.apache.org/viewcvs.cgi/httpd-test/perl-framework/README?rev=1.8&content-type=text/vnd.viewcvs-markup

                                      _____________________________________________________________________
                                      Stas Bekman JAm_pH -- Just Another mod_perl Hacker
                                      http://stason.org/ mod_perl Guide http://perl.apache.org/guide
                                      mailto:stas@... http://ticketmaster.com http://apacheweek.com
                                      http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
                                    Your message has been successfully submitted and would be delivered to recipients shortly.