Loading ...
Sorry, an error occurred while loading the content.

RE: Waterfall (was RE: [XP] Project help)

Expand Messages
  • Blum, Robert
    ... [Re: Royce Waterfall paper] ... Yes, it makes for quite an interesting read. Obviously, something has gone wrong with the interpretation of Waterfall early
    Message 1 of 6 , Apr 1, 2002
    • 0 Attachment
      > From: Laurent Bossavit [mailto:laurent@...]

      [Re: Royce Waterfall paper]

      > Very, very interesting article. The strange thing is, it defines a
      > process which is almost, but not quite, entirely unlike Waterfall as
      > that is usually discussed.

      Yes, it makes for quite an interesting read. Obviously, something has gone
      wrong with the interpretation of Waterfall early on....

      > The first notable point is how things go wrong at a very early stage.
      > Royce starts with, "There are two essential steps common to all
      > computer program developments [...] first an analysis step, followed
      > second by a coding step".
      >
      > Now, I'm well aware that the meanings of "analysis" and "coding" have
      > shifted in subtle ways over the intervening three decades. But it
      > does seem to me that Royce here commits what might be the Original
      > Sin of software development : forgetting that using software is also
      > a way of testing it, and that "testing" certainly is one of these
      > "essential" steps.


      Well, Royce is saying that a 'small computer program for internal
      operations' is created that way. Which, in a way, is true. The scope of
      small might have changed, but usually small internal tools receive exactly
      as much attention as is needed to limp through the finish line.

      He is in fact saying that those efforts are only enough 'if the effort is
      sufficiently small and if the final product is to be operated by those who
      built it'. And as we all know, even here on this list, we sometimes skip
      testing if we think it's small enough.

      > The next interesting thing is that Royce then draws the "classic"
      > picture of the Waterfall cycle, then immediately goes on to say "I
      > believe in this concept, but the implementation described above is
      > risky and invites failure". He then goes on to describe most of what
      > is wrong with Waterfall.

      Yes. And this is what finally has brought home to me the point that you
      cannot talk to management as you would talk to an engineer. I am (really!)
      convinced that what happened was more or less the following:

      Engineer: I just heard of this paper on process, and it describes what a
      good process should look like. Very interesting.
      Manager: Well, what is it about? Can I see it?
      [Engineer makes a copy. Hands it to manager. Back to punching paper tape]
      [Manager reads it. First diagram clearly not applicable, even mentions
      'small application'. Second diagram says 'Implementation steps to develop a
      large computer program for delivery to a customer'.]
      Manager: This is a brilliant paper! Implement this process!

      Yes, I'm a bit cynical. But I also believe this more or less to be true. Not
      because I think managers are all incompetent - they are not. But they DO
      have different priorities and modes of communication than an engineer. So
      'Original Sin' (I love that!) for me happens where the engineer hands a
      simple copy to the manager. Had he delivered a summary instead, the world
      might look different today. (Well, maybe not. But one can dream)

      > He outlines an "executive plan" of five critical remedies for the
      > weaknesses in the simpler Waterfall picture.

      Yes, after presenting a diagram that at a first glance looks like it
      describes 'the right thing'. This is where the breakdown happens - if you're
      looking only for a solution to your process problem, you might well be
      inclined to stop at this diagram. Moral: If you give a presentation, make
      sure it communicates the right thing, even if quoted out of context.

      > After applying these
      > remedies, the final picture looks nothing like the original; and
      > certainly nothing like Waterfall as it is usually portrayed.
      >
      > It is interesting to quote the "executive plan" in full :
      > - Complete program design before analysis and coding begins

      Ah yes, the BDUF. Keep in mind that back in the day it was actually
      expensive to write code. No fancy IDE, no nothing. Submit your batch for a
      run, wait for the data center to process it, get the results back. Several
      days turnaround for a change. In this environment it actually _is_ cheaper
      to BDUF.

      Royce even recognizes this fact, if not explicitly. 'Until coding begins
      these three nouns (documentation, specification, design) denote a single
      thing.'

      Furthermore, he sees the real (monetary) value of design _after_ the program
      is created.
      - during testing. Royce sees the value of the documentation in the fact that
      _every_ team member can analyze mistakes. Otherwise, the original 'owner'
      would have to fix the problem, since only he understands the area. Clearly,
      he's opposed to code ownership...

      And, to make things more interesting, he saw that this might be done on a
      computer, and recommends against it because it is too expensive.

      - during the operational phase. 'Without good documentation, the software
      must be operated by those who built it'. Which still holds true today.

      - for updating/retrofitting. Again, some (for me) surprising insights: 'If
      documentation does not exist, generally the entire exisiting framework of
      operating software must be junked, even for relatively modest changes.'.
      Read all the threads about legacy code to find out how true this is...

      The difference is only that we are fortunate enough to be _able_ to document
      by writing test cases. If our unit tests could only be run once every few
      days (and we had to actually deliver syntactically correct code the first
      time round :), I do think we would still be writing large reams of paper.

      > - Documentation must be current and complete
      Replace 'Documentation' with 'Customer and programmer tests'...

      > - Do the job twice if possible
      'insofar as critical design/operation areas are concerned'. He goes on to
      say that this can actually be done as a simulation, 'to a time scale that is
      relatively small with respect to the overall effort'. I don't know about
      you, but that (again in the context of programming way back...) sounds
      surprisingly like 'Spike the things you are not sure about'.

      Another interesting idea from that section is that this simulation actually
      gives a basis for judgment - without it, 'the project manager is at the
      mercy of human judgment'. Does that sound like 'don't speculate, test', or
      'Let the code tell you' to you?

      > - Testing must be planned, controlled and monitored
      And XP achieves the ultimate form of that. Our testing is _very_ well
      planned. It is the plan. It is constantly controlled and monitored.

      > - Involve the customer
      Arg. Of all the things people left out in implementing waterfall, this hurts
      the most. It was plain there, 30 years ago...

      > When I read the article, at one point I did a double-take. "Eh ? Do
      > *design* before analysis ?" Not a single description of Waterfall
      > that I had read before the Royce paper so much as mentioned this
      > possibility. McConnell gets it wrong. From Googling around a bit, I
      > get the impression that *everyone* gets it wrong.

      Which is really interesting. I just checked, and McConnell does not quote
      Royce at all. Checking out Citeseer, _NOBODY_ quoted Royce up until 1987 or
      so. (Or Citeseer simply does not go back further...)

      I wonder what the original paper was that described waterfall as we know
      it...

      > On the other hand, I do now know where the Almighty Thump came from.
      > Royce recommended that at least 1200 pages of specification
      > documentation be written for a $5 million software proposal. And
      > that's just one part of the documentation. :)

      Yes, but again, let's put it back in context. Documentation was a
      replacement for quick compile/test cycles. If we look at all our tests, how
      much documentation is that? Now, if we wrote them in plain english, and not
      in (insert favorite language here), how much text would that be?

      > It's not overly surprising that Royce emphasizes both prototyping (Do
      > It Twice) and testing.

      Well, prototyping came as quite a surprise to me. Based on how Waterfall is
      usually described, that was not something I expected to pop up.

      > Although these aspects are the ones that most
      > systematically get forgotten in "real life" software efforts... More
      > surprising is the final advocation to "involve the customer"... But
      > then Royce only allots one paragraph to that idea ! Maybe that's the
      > reason why it's taken 30 years to rediscover the value of that...

      Well, given that the whole article is only 5 or so pages of text, a
      paragraph seems quite a lot. The real problem is that nobody followed up.

      This paper makes it very clear where Waterfall came from, and that it was
      not that bad an idea - originally. We just forgot to keep monitoring the
      constraints that made Waterfall a good idea.

      - Robert
    Your message has been successfully submitted and would be delivered to recipients shortly.