Loading ...
Sorry, an error occurred while loading the content.

Re: [extremeperl] Re: Testing, Audience, etc.

Expand Messages
  • Ged Haywood
    Hi all, ... [snip] ... See attached. I once had to fly to seven or eight different countries over a period of several weeks to fix a $0.25 problem in a couple
    Message 1 of 20 , Feb 7, 2002
    • 0 Attachment
      Hi all,

      On Thu, 7 Feb 2002, Drew Taylor wrote:

      > At 11:45 PM 2/6/2002 -0700, chromatic wrote:
      [snip]
      > managements edicts & timelines that forces the lack of tests. As for the
      > last point, I read somewhere (perhaps in one of Steve McConnell's books)
      > that a study found it takes magnitudes of more time/money to fix a bug
      > after the fact that to fix it before. And if it negatively affects a
      > customer, then multiply that cost by several factors for pissed off said
      > customer. If only management could be made to understand this fact...

      See attached.

      I once had to fly to seven or eight different countries over a period
      of several weeks to fix a $0.25 problem in a couple of hundred $22,000
      instruments because the guy in procurement had ignored my written
      procurement specification and the guy in test had ignored my written
      test secification. The problems only started to surface when the
      instruments were used in hot places.

      It still bugs me that I didn't send a bill to their employer, who was
      my supplier and contracted to make the things to the specification.

      It's called experience.

      73,
      Ged.


      [Non-text portions of this message have been removed]
    • Rob Nagler
      ... The first step is to make sure your interface has structure. If you are testing arbitrarily constructed templates, you re going to have a rough time. If
      Message 2 of 20 , Feb 8, 2002
      • 0 Attachment
        > How do you do that? I've only heard of tools that allow you to create a
        > script and then playback that script. This is semiuseful, but what happens
        > if you make a change to the interface?

        The first step is to make sure your interface has structure. If you
        are testing arbitrarily constructed templates, you're going to have a
        rough time. If you build your HTML pages from widgets or
        parameterized templates, you'll have some structure to grab on to.

        > What tools/techniques have you used
        > in the past to do acceptance testing?

        The best tool is perl. We used it to test our CORBA based Web server,
        and we use it to test our application written in perl. It is not very
        hard to build an acceptance test suite using tools like LWP and
        HTMLParser.

        I am a little behind schedule. My goal is to release our internal
        infrastructure by the end of the month. It will come with a test
        suite which tests our pet shop demo (http://petshop.bivio.biz).

        > Do I have to setup a fake web server
        > environment & run the tests that way?

        I find for acceptance testing you need a test environment which is as
        close to your production environment as possible. Any of our
        developers can run the test suite on their personal Web servers.
        Every night we run the test suite against our test servers, which are
        relatively clean machines.

        > good. What I'm ultimately interested in learning is better design. Learning
        > patterns is one step, and working to see the problem from a higher level
        > view are two things I'm doing now.

        To me, there are two sides: analysis and synthesis. Patterns are
        about synthesis. XP is about analysis. I'm not a big fan of
        patterns, because they're very focused on classical object-oriented
        programming, and I try to program declaratively whenever I can. In
        addition, languages like Java, have some serious deficiencies such as
        weak ability to delegate and no class level inheritance.

        I find reading books about Lisp, functional programming, and logic
        programming expands my solution set much better than reading a book
        about design patterns.

        > OK. Why don't they just say that? :-) I know that at high levels it's
        > essential that we're all speaking the same language, but can't that
        > language be simpler?

        It is really hard to write cogent prose, which addresses a wide
        audience. They're just some concepts which are hard to explain in
        simpler language. I'm reading a book by Einstein which is incredibly
        well written but I have a really hard time understanding his
        discussions about the special theory and general theory of
        relativity. That's why I'm a programmer, I guess.

        > anyway. Besides, they're probably a little afraid of all the little bugs it
        > might turn up. ;-)
        >
        > Could this be a hidden reason some shops are wary of tests?

        I don't think so. In general, people have a hard time quantifying
        quality. If it works in the general case, it may be enough. The user
        base may be small. I really like Gerry Weinberg's comments on quality
        in his book "Quality Software Management: Vol. 1 Systems Thinking":

        The Quality Statement: Every statement about quality is a statement about some
        person(s).

        The Political Dilemma: More quality for one person may mean less
        quality for another.

        The Quality Decision: Whose opinion of quality is to count when making
        decisions?

        The Inadequate Definition of Quality: Quality is the absence of error.

        The Absence of Errors Fallacy: Though copious errors guarantees
        worthlessness, having zero errors guarantees nothing at all about the
        value of software.

        I highly recommend the book.

        Rob
      • Stas Bekman
        ... Since you are talking about testing apps against webserver, I d plug in the new Apache::Test framework which most Apache::* modules and frameworks
        Message 3 of 20 , Feb 21, 2002
        • 0 Attachment
          Rob Nagler wrote:

          >>What tools/techniques have you used
          >>in the past to do acceptance testing?
          >>
          >
          > The best tool is perl. We used it to test our CORBA based Web server,
          > and we use it to test our application written in perl. It is not very
          > hard to build an acceptance test suite using tools like LWP and
          > HTMLParser.
          >
          > I am a little behind schedule. My goal is to release our internal
          > infrastructure by the end of the month. It will come with a test
          > suite which tests our pet shop demo (http://petshop.bivio.biz).
          >
          >
          >>Do I have to setup a fake web server
          >>environment & run the tests that way?
          >>
          >
          > I find for acceptance testing you need a test environment which is as
          > close to your production environment as possible. Any of our
          > developers can run the test suite on their personal Web servers.
          > Every night we run the test suite against our test servers, which are
          > relatively clean machines.

          Since you are talking about testing apps against webserver, I'd plug in
          the new Apache::Test framework which most Apache::* modules' and
          frameworks' developers will find very helpful. The goal is to have every
          Apache::* module needing mod_perl or just plain apache env, use
          Apache::Test for its test. There is no more excuses for not having
          tests. And if something is missing from its functionality now it's the
          time to jump in and ask for it/add it.

          httpd-test project is using this Perl framework for testing C modules
          for Apache 1.3 and 2.0 and the server itself. And of course originally
          it was developed for mod_perl 2.0. The same test suite can work with
          httpd 1.3 and httpd 2.0. For more info see:
          http://perl.apache.org/preview/modperl-site/docs/2.0/devel/testing/testing.html

          Once we release the new modperl site (hopefully in a few weeks) this URL
          will appear as:
          http://perl.apache.org/docs/2.0/devel/testing/testing.html

          I've started mentioning XP in this doc and mention reasons for a need to
          test, but more work in needed so your help is very welcome.

          To get the framework grab the httpd-test rep from cvs or the snapshot:
          http://cvs.apache.org/snapshots/modperl-2.0/

          also see:
          http://cvs.apache.org/viewcvs.cgi/httpd-test/perl-framework/README?rev=1.8&content-type=text/vnd.viewcvs-markup

          _____________________________________________________________________
          Stas Bekman JAm_pH -- Just Another mod_perl Hacker
          http://stason.org/ mod_perl Guide http://perl.apache.org/guide
          mailto:stas@... http://ticketmaster.com http://apacheweek.com
          http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
        Your message has been successfully submitted and would be delivered to recipients shortly.