Loading ...
Sorry, an error occurred while loading the content.

What's wrong with tracking estimates vs. actuals?

Expand Messages
  • Daniel Gackle
    At my company, some managers believe in tracking estimates vs. actuals. They like what my team is doing with Scrum, but they d like it better if we gave them a
    Message 1 of 20 , Oct 4, 2003
    • 0 Attachment
      At my company, some managers believe in tracking estimates vs. actuals. They
      like what my team is doing with Scrum, but they'd like it better if we gave
      them a spreadsheet every month matching the estimate for each task with the
      hours actually spent on that task.

      I feel bad about this and don't want to do it. It feels contrary to the
      spirit of self-organizing teams, like someone is looking over our shoulder.
      Yet I can't fully articulate what's wrong with it.

      Can anyone help me get clearer on this?

      - Daniel

      -----Original Message-----
      From: "Gamble, Ken" <ken.gamble@...>
      Subject: New Scrum Article Available

      No matter how well someone measures past estimates against actuals even a
      small change in the estimate can have a big effect on the outcome of a
      chaotic/complex process no matter how good the model is or the resolution of
      the measurements.

      What this means for software development is that even though we can use
      previous estimates as part of a process model for delivering software we
      have to keep a sharp eye (Scrum management) on the process because it can
      wonder off course simply because of these small values, even if the process
      model is very tightly defined.
    • Dan Rawsthorne
      It would be a good idea to track the total estimate versus the total actuals, but not at the task level. What makes the totals work out is the law of large
      Message 2 of 20 , Oct 4, 2003
      • 0 Attachment
        It would be a good idea to track the total estimate versus the total
        actuals, but not at the task level. What makes the totals work out is
        the law of large numbers - that everything averages out. What your boss
        should be hoping for is that your estimates average out to be correct.
        It is unreasonable to expect that every individual estimate will be
        correct, and trying to get that to happen always leads to analysis
        paralysis. However, making sure that your estimates are converging to
        actuals at the iteration level makes sense to me.

        Dan ;-)

        Dan Rawsthorne, PhD, Sr. Consultant
        www.netobjectives.com
        DrDan@...
        office: 425-269-8628

        Net Objectives' vision is effective software development without
        suffering. Our mission is to assist software development teams in
        accomplishing this through a combination of training and mentoring.


        > -----Original Message-----
        > From: Daniel Gackle [mailto:gackle@...]
        > Sent: Saturday, October 04, 2003 11:35 PM
        > To: scrumdevelopment@yahoogroups.com
        > Subject: [scrumdevelopment] What's wrong with tracking estimates vs.
        > actuals?
        >
        > At my company, some managers believe in tracking estimates vs.
        actuals.
        > They
        > like what my team is doing with Scrum, but they'd like it better if we
        > gave
        > them a spreadsheet every month matching the estimate for each task
        with
        > the
        > hours actually spent on that task.
        >
        > I feel bad about this and don't want to do it. It feels contrary to
        the
        > spirit of self-organizing teams, like someone is looking over our
        > shoulder.
        > Yet I can't fully articulate what's wrong with it.
        >
        > Can anyone help me get clearer on this?
        >
        > - Daniel
        >
        > -----Original Message-----
        > From: "Gamble, Ken" <ken.gamble@...>
        > Subject: New Scrum Article Available
        >
        > No matter how well someone measures past estimates against actuals
        even a
        > small change in the estimate can have a big effect on the outcome of a
        > chaotic/complex process no matter how good the model is or the
        resolution
        > of
        > the measurements.
        >
        > What this means for software development is that even though we can
        use
        > previous estimates as part of a process model for delivering software
        we
        > have to keep a sharp eye (Scrum management) on the process because it
        can
        > wonder off course simply because of these small values, even if the
        > process
        > model is very tightly defined.
        >
        >
        > ------------------------ Yahoo! Groups Sponsor
        >
        > To Post a message, send it to: scrumdevelopment@...
        > To Unsubscribe, send a blank message to: scrumdevelopment-
        > unsubscribe@...
        >
        > Your use of Yahoo! Groups is subject to
        http://docs.yahoo.com/info/terms/
        >
        >
        >
        >
        ________________________________________________________________________
        > This email has been scanned for all viruses by the MessageLabs Email
        > Security System. For more information on a proactive email security
        > service working around the clock, around the globe, visit
        > http://www.messagelabs.com
        >
        ________________________________________________________________________
      • Ken Schwaber
        Dan, It s not posted yet, but someone reminded me of Holland s description of a complex process or object. It is something of which the only model is itself;
        Message 3 of 20 , Oct 5, 2003
        • 0 Attachment
          Dan,
          It's not posted yet, but someone reminded me of Holland's description of a complex process or object. It is something of which the only model is itself; there is no abstraction of it that is possible. That's the core problem with comparing estimates to actuals; there is no abstraction that will improve the estimating in the future. Your managers are thinking of simple processes, not complex processes.
          Ken
          >
          > From: Daniel Gackle <gackle@...>
          > Date: 2003/10/05 Sun AM 01:35:20 CDT
          > To: scrumdevelopment@yahoogroups.com
          > Subject: [scrumdevelopment] What's wrong with tracking estimates vs. actuals?
          >
          > At my company, some managers believe in tracking estimates vs. actuals. They
          > like what my team is doing with Scrum, but they'd like it better if we gave
          > them a spreadsheet every month matching the estimate for each task with the
          > hours actually spent on that task.
          >
          > I feel bad about this and don't want to do it. It feels contrary to the
          > spirit of self-organizing teams, like someone is looking over our shoulder.
          > Yet I can't fully articulate what's wrong with it.
          >
          > Can anyone help me get clearer on this?
          >
          > - Daniel
          >
          > -----Original Message-----
          > From: "Gamble, Ken" <ken.gamble@...>
          > Subject: New Scrum Article Available
          >
          > No matter how well someone measures past estimates against actuals even a
          > small change in the estimate can have a big effect on the outcome of a
          > chaotic/complex process no matter how good the model is or the resolution of
          > the measurements.
          >
          > What this means for software development is that even though we can use
          > previous estimates as part of a process model for delivering software we
          > have to keep a sharp eye (Scrum management) on the process because it can
          > wonder off course simply because of these small values, even if the process
          > model is very tightly defined.
          >
          >
          >
          > To Post a message, send it to: scrumdevelopment@...
          > To Unsubscribe, send a blank message to: scrumdevelopment-unsubscribe@...
          >
          > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
          >
          >
          >
        • Mike Cohn
          I d say you shouldn t do it because it doesn t add value commensurate with its cost. Don t argue with your bosses that it adds no value because comparing
          Message 4 of 20 , Oct 5, 2003
          • 0 Attachment
            I'd say you shouldn't do it because it doesn't add value commensurate with
            its cost. Don't argue with your bosses that it "adds no value" because
            comparing what you originally thought a task would take with what it did
            take can help make you a better estimator.

            But, it can be time-consuming to track actuals, especially for a full team
            where some on the team are probably already decent estimators.

            Because Scrum already has solid mechanisms for monitoring whether all the
            work gets done in a sprint (high team commitment, daily burndown charts,
            daily scrum, and so on), Scrum does not have the same reliance on early and
            accurating estimating that a predictive or waterfall approach does.

            So--the cost to gather actuals is the same in Scrum or waterfall. The
            benefit in Scrum is greatly reduced.

            --Mike



            > -----Original Message-----
            > From: Daniel Gackle [mailto:gackle@...]
            > Sent: Sunday, October 05, 2003 12:35 AM
            > To: scrumdevelopment@yahoogroups.com
            > Subject: [scrumdevelopment] What's wrong with tracking estimates vs.
            > actuals?
            >
            > At my company, some managers believe in tracking estimates vs. actuals.
            > They
            > like what my team is doing with Scrum, but they'd like it better if we
            > gave
            > them a spreadsheet every month matching the estimate for each task with
            > the
            > hours actually spent on that task.
            >
            > I feel bad about this and don't want to do it. It feels contrary to the
            > spirit of self-organizing teams, like someone is looking over our
            > shoulder.
            > Yet I can't fully articulate what's wrong with it.
            >
            > Can anyone help me get clearer on this?
            >
            > - Daniel
            >
            > -----Original Message-----
            > From: "Gamble, Ken" <ken.gamble@...>
            > Subject: New Scrum Article Available
            >
            > No matter how well someone measures past estimates against actuals even a
            > small change in the estimate can have a big effect on the outcome of a
            > chaotic/complex process no matter how good the model is or the resolution
            > of
            > the measurements.
            >
            > What this means for software development is that even though we can use
            > previous estimates as part of a process model for delivering software we
            > have to keep a sharp eye (Scrum management) on the process because it can
            > wonder off course simply because of these small values, even if the
            > process
            > model is very tightly defined.
            >
            >
            >
            > To Post a message, send it to: scrumdevelopment@...
            > To Unsubscribe, send a blank message to: scrumdevelopment-
            > unsubscribe@...
            >
            > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
          • Marc Hamann
            ... What are they trying to improve with this metric? Make the team better estimators? The bound on estimation is knowledge: the more you know about what you
            Message 5 of 20 , Oct 5, 2003
            • 0 Attachment
              At 02:35 AM 10/5/03, Daniel wrote:
              >At my company, some managers believe in tracking estimates vs. actuals.

              What are they trying to improve with this metric? Make the team better
              estimators?

              The bound on estimation is knowledge: the more you know about what you
              have to do, the better you can estimate.

              This means that if you are doing something you've done before in exactly
              the same way, you can estimate it better than for something you haven't.

              But the whole idea of being agile is to be able to respond to the unknown
              changes that inevitably come. Encouraging people to become better
              estimators is equivalent to asking them to take fewer risks and to be in
              denial about radical changes in requirements and circumstances. Dealing
              with those things means throwing your estimates off.

              So why measure it?

              What I WOULD want to measure is effectiveness: is the team delivering more
              functionality per sprint (especially over the first several sprints), or at
              least an acceptable constant level. Delivered functionality is the only
              metric that matters, IMHO, so spending time and "guilt points" on any other
              metric that isn't directly dependent on this is counter-productive.

              You'll simply encourage "improvement" in a direction that is not aligned
              with your primary goal.

              Hope this is an argument you can use. ;-)

              Marc
            • David J. Anderson
              I m glad this topic came up because I ve been thinking about it a lot recently. Thanks Daniel! In FDD as documented by De Luca, Palmer and Felsing, developers
              Message 6 of 20 , Oct 5, 2003
              • 0 Attachment
                I'm glad this topic came up because I've been thinking
                about it a lot recently. Thanks Daniel!

                In FDD as documented by De Luca, Palmer and Felsing,
                developers are asked for planned and actual dates for
                each of the 6 Feature milestones. In my own very
                recent book, "Agile Management for Software
                Engineering", http://www.agilemanagement.net/ , I
                document a different - more Scrum-like approach and
                described why I changed FDD and why I believed the
                alternative worked better.

                I abandonded planned dates on individual Feature
                milestones and replaced them with a single date for
                delivery of a Chief Programmer Work Package (an
                approximately 2 week batch of work) along with a
                scrum-like daily standup meeting. The team was asked
                to focus daily on the production rate (equivalent to
                burndown rate). With the team I was managing in Kansas
                City, at the time, this worked much better. I
                documented this in The Coad Letter #101,
                http://bdn.borland.com/article/0,1410,29686,00.html

                It eliminated the local safety problem of individual
                commitments, i.e. too much buffer on each fine-grained
                task. It also [as Mike C] points out eliminates cost
                of calculating the planned dates. In FDD the cost of
                tracking the actual deliveries is trivial.

                However, a Development Manager - Mike Watson - that I
                work with currrently in Seattle, is busy changing his
                team back to Jeff De Luca's preferred approach of
                individual milestone commitments. Why?

                We have concluded that whether or not the aggregated
                commitment and daily standup works best versus planned
                individual milestones has a lot to do with the culture
                of the organization and the mindset or attitude of the
                team. In the most recent case, the daily standup
                meeting was proving ineffective for reasons too
                complex to explain in this post. Also the team
                mentality tended towards another time delivery problem
                - student syndrome - the idea that a task is started
                at the last available moment and any safety buffer is
                used up first through procrastination. Hence, even on
                a 2-week commitment the team wouldn't get going at
                full speed until closer to the completion date. The
                result was that often the completion date was missed.
                By changing back to fine-grained planned versus actual
                dates, this problem was eliminated.

                Hence, I now feel that both solutions work under
                certain conditions and circumstances, with different
                sets of people and different management cultures.

                Regards,

                David
                --
                David J. Anderson
                author of "Agile Management for Software Engineering"
                http://www.agilemanagement.net/


                --- Mike Cohn <mike@...> wrote:

                I'd say you shouldn't do it because it doesn't add
                value commensurate with
                its cost. Don't argue with your bosses that it "adds
                no value" because
                comparing what you originally thought a task would
                take with what it did
                take can help make you a better estimator.

                But, it can be time-consuming to track actuals,
                especially for a full team
                where some on the team are probably already decent
                estimators.

                Because Scrum already has solid mechanisms for
                monitoring whether all the
                work gets done in a sprint (high team commitment,
                daily burndown charts,
                daily scrum, and so on), Scrum does not have the same
                reliance on early and
                accurating estimating that a predictive or waterfall
                approach does.

                So--the cost to gather actuals is the same in Scrum or
                waterfall. The
                benefit in Scrum is greatly reduced.

                --Mike




                __________________________________
                Do you Yahoo!?
                The New Yahoo! Shopping - with improved product search
                http://shopping.yahoo.com
              • Marc Hamann
                ... I don t think this is a planning problem but engineering practices problem. If the team is failing to meet their commitments due to procrastination, you
                Message 7 of 20 , Oct 5, 2003
                • 0 Attachment
                  At 03:35 PM 10/5/03, David J. Anderson wrote:
                  >Also the team
                  >mentality tended towards another time delivery problem
                  >- student syndrome - the idea that a task is started
                  >at the last available moment and any safety buffer is
                  >used up first through procrastination.

                  I don't think this is a planning problem but "engineering practices" problem.

                  If the team is failing to meet their commitments due to procrastination,
                  you have to ask yourself if you have the team you need.
                  If you decide not to fire them immediately, you can use the daily meeting
                  to expose that progress is not being made early in the sprint, and to push
                  them to use techniques that produce more regular increments of value. (One
                  such technique is test-driven development.)

                  As an alternative, reduce the length of your sprint until they don't feel
                  there is any buffer.

                  I think that baby-sitting the team into behaving like professionals through
                  micro-measurements can only have deleterious effects in the long run, and
                  will neutralize the benefits of using an agile development methodology.

                  If on the other hand, the team IS meeting its commitments, in spite of
                  procrastinating, let them procrastinate, I say. Maybe that just works
                  better for them, and so be it.

                  Either way, you encourage the wrong adaptations by instituting an
                  estimation improvement regime.

                  Marc
                • Ron Jeffries
                  ... Probably not, but I ll try. Some folks have touched on some of these thoughts already. My initial reaction is that for some reason, someone doesn t trust
                  Message 8 of 20 , Oct 5, 2003
                  • 0 Attachment
                    On Sunday, October 5, 2003, at 2:35:20 AM, Daniel Gackle wrote:

                    > At my company, some managers believe in tracking estimates vs. actuals. They
                    > like what my team is doing with Scrum, but they'd like it better if we gave
                    > them a spreadsheet every month matching the estimate for each task with the
                    > hours actually spent on that task.

                    > I feel bad about this and don't want to do it. It feels contrary to the
                    > spirit of self-organizing teams, like someone is looking over our shoulder.
                    > Yet I can't fully articulate what's wrong with it.

                    > Can anyone help me get clearer on this?

                    Probably not, but I'll try. Some folks have touched on some of these
                    thoughts already.

                    My initial reaction is that for some reason, someone doesn't trust you. I
                    hope that's not the case, but if it is, I'd try to figure out some way to
                    deal with it directly.

                    Second, it probably really doesn't do any harm if you collect this
                    information. They might try to use it to set objectives to make the numbers
                    match. If they did, I'd respond thusly:

                    1. Seems to me that what's important is that the project is on schedule.
                    (If it is. If it isn't, that's what's important.) When we schedule things
                    into a Sprint, the team looks at the whole picture, not just the
                    individual tasks, and rallies around to get the best possible combination
                    of software does that meets the Sprint Goal. That's what's making this
                    work. We don't really even know who is going to work on some of these
                    tasks, until the time comes to do it. The purpose of the estimates is to
                    be able to add them up and come close enough to a Sprint Goal that can be
                    attained. We're not trying to get them "right", we're trying to use them
                    to estimate the overall cost of gaining the business value represented by
                    the next Sprint Goal. Here's how well they're working for that <drawing
                    their attention back to the point of how the whole project is going>.

                    2. When we look at these numbers, we see two things: we see a general
                    (under/over) estimate component, that looks like about (X) percent. And
                    around that systematic error, we see a random component that is about (Y)
                    percent. So long as the random component is reasonable (and it is (unless
                    it isn't)), the laws of large numbers cancel the errors out. That's why
                    the burndown chart looks so straight <drawing their attention back to how
                    well burndown is going>.

                    3. We are concerned about systematic error. [At least I guess that Scrum
                    must be concerned about that. XP is not.] Here's how it is varying over
                    time. Each Sprint we look at how we did and we adjust our estimates
                    basically proportionately, taking into account anything we've leared
                    about specific requirements that might make their estimates go up or down
                    relative to the others.

                    4. The most important thing, boss, is this: we're new at Scrum. Scrum has
                    been designed by people who are much smarter than we are about these
                    things, and it has been tested in many situations like ours. Until we get
                    more experience with doing the process out of the box, chief, I recommend
                    that we stick with the basic Scrum. We should take a look at making it
                    better once we can do it in its standard form. After all, it's delivering
                    us good results.

                    Or something like that.

                    Of course, having collected the information and showed it to them, we might
                    find out that it's completely harmless to do so. That's always possible ...

                    Ron Jeffries
                    www.XProgramming.com
                    Master your instrument, master the music,
                    and then forget all that *!xy!@ and just play. -- Charlie Parker
                  • Anko Tijman
                    ... wrote: Your managers are thinking of simple processes, not complex processes. ... Traditional approach: thinking of a simple process,
                    Message 9 of 20 , Oct 6, 2003
                    • 0 Attachment
                      --- In scrumdevelopment@yahoogroups.com, Ken Schwaber
                      <ken.schwaber@v...> wrote:
                      Your managers are thinking of simple processes, not complex processes.
                      > Ken

                      Traditional approach: thinking of a simple process, and developing
                      complex software

                      Agile approach: thinking of a complex process, and developing simple
                      software



                      What a contradiction....
                      ;-)


                      Anko Tijman
                    • David J. Anderson
                      Marc, It s always interesting to speculate on what works and doesn t work. In this case, I know that both approaches work with different teams. I have run more
                      Message 10 of 20 , Oct 6, 2003
                      • 0 Attachment
                        Marc,

                        It's always interesting to speculate on what works and
                        doesn't work.

                        In this case, I know that both approaches work with
                        different teams. I have run more than 15 projects in 4
                        Fortune 1000 companies (more than 10 of them at Sprint
                        and Motorola - both Fortune 100).

                        Hence, I just don't buy into the "one way works and
                        the other way doesn't" belief. I have the metrics and
                        the released systems to prove it.

                        You're suggested approach of "just fire them" is
                        seldom an option for a manager in a big company - nor
                        should it be. Getting people to change can be
                        difficult or impossible - changing a culture can be
                        harder.

                        Regards,

                        David

                        --- Marc Hamann <marc@...> wrote:

                        I think that baby-sitting the team into behaving like
                        professionals through
                        micro-measurements can only have deleterious effects
                        in the long run, and
                        will neutralize the benefits of using an agile
                        development methodology.

                        If on the other hand, the team IS meeting its
                        commitments, in spite of
                        procrastinating, let them procrastinate, I say. Maybe
                        that just works
                        better for them, and so be it.

                        Either way, you encourage the wrong adaptations by
                        instituting an
                        estimation improvement regime.

                        Marc



                        =====
                        David J Anderson
                        Author of "Agile Management for Software Engineering"
                        http://www.agilemanagement.net/

                        __________________________________
                        Do you Yahoo!?
                        The New Yahoo! Shopping - with improved product search
                        http://shopping.yahoo.com
                      • Marc Hamann
                        David, My claim is not that your approach doesn t work; there a innumerable systems that have been built using BDUF or even ad hoc methodologies. I m sure that
                        Message 11 of 20 , Oct 6, 2003
                        • 0 Attachment
                          David,

                          My claim is not that your approach doesn't work; there a innumerable
                          systems that have been built using BDUF or even ad hoc methodologies.
                          I'm sure that many of these conform to various sorts of acceptable metrics.

                          However, such practices are not Agile in general or Scrum in
                          particular. Moreover, they work against the goals and values that Agile
                          and Scrum promote, which is counter-productive if you are trying to combine
                          them with Agile practices.

                          My implication that the team should be fired was provocative, but "that
                          won't work here because of our culture/our programmers/our boss" is the
                          most common objection I hear to using Agile techniques. This seems
                          counter-intuitive to me, since I believe that the great virtue of Agile
                          techniques is that they can be used even in adverse circumstances.

                          You don't have to change the culture to start using the techniques; just do
                          your best to implement the practices, and avoid any practices which work
                          against the goals and values you are trying to promote.

                          It's not magic, it's not easy, but if you don't at least try the best you
                          can, you can't claim to be doing Scrum or Agile.

                          Regards,

                          Marc

                          At 01:12 PM 10/6/03, you wrote:
                          >Marc,
                          >
                          >It's always interesting to speculate on what works and
                          >doesn't work.
                          >
                          >In this case, I know that both approaches work with
                          >different teams. I have run more than 15 projects in 4
                          >Fortune 1000 companies (more than 10 of them at Sprint
                          >and Motorola - both Fortune 100).
                          >
                          >Hence, I just don't buy into the "one way works and
                          >the other way doesn't" belief. I have the metrics and
                          >the released systems to prove it.
                          >
                          >You're suggested approach of "just fire them" is
                          >seldom an option for a manager in a big company - nor
                          >should it be. Getting people to change can be
                          >difficult or impossible - changing a culture can be
                          >harder.
                          >
                          >Regards,
                          >
                          >David
                          >
                          >--- Marc Hamann <marc@...> wrote:
                          >
                          >I think that baby-sitting the team into behaving like
                          >professionals through
                          >micro-measurements can only have deleterious effects
                          >in the long run, and
                          >will neutralize the benefits of using an agile
                          >development methodology.
                          >
                          >If on the other hand, the team IS meeting its
                          >commitments, in spite of
                          >procrastinating, let them procrastinate, I say. Maybe
                          >that just works
                          >better for them, and so be it.
                          >
                          >Either way, you encourage the wrong adaptations by
                          >instituting an
                          >estimation improvement regime.
                          >
                          >Marc
                          >
                          >
                          >
                          >=====
                          >David J Anderson
                          >Author of "Agile Management for Software Engineering"
                          >http://www.agilemanagement.net/
                          >
                          >__________________________________
                          >Do you Yahoo!?
                          >The New Yahoo! Shopping - with improved product search
                          >http://shopping.yahoo.com
                          >
                          >
                          >To Post a message, send it to: scrumdevelopment@...
                          >To Unsubscribe, send a blank message to:
                          >scrumdevelopment-unsubscribe@...
                          >
                          >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                        • David J. Anderson
                          Marc, I think there is a danger of doing agile for the sake of it. The real goal is to make more money by delivering more value faster to the end of value
                          Message 12 of 20 , Oct 6, 2003
                          • 0 Attachment
                            Marc,

                            I think there is a danger of "doing agile" for the
                            sake of it. The real goal is to make more money by
                            delivering more value faster to the end of value
                            chain. If agilists lose sight of this then agile will
                            be a fad rather than a genuine trend.

                            I would contest your observation that making the best
                            ROI or the most money is always best done using the
                            very pure agile techniques you describe.

                            For example, in the Boehm and Turner book they
                            describe a 3 year long XP project which using more
                            than 30 developers in a 50 person team produced around
                            500,000 SLOC. Compare this to the CLS project at UOB
                            in Singapore (the original FDD project) which was
                            performed using the fine-grained planned versus actual
                            which with slightly less people (47) but with only 23
                            developers produced 1,500,000 SLOC in only 18 months.

                            If SLOC is a reliable metric (and I don't believe it
                            is but given a lack of a function point assessment of
                            both projects it is all we have) then the first FDD
                            project seems to be up to 6 times more effective than
                            the textbook XP project reported by Boehm and Turner.

                            Not only was the CLS project a huge success but after
                            it was rolled live - the bank (UOB) was able to take
                            over its nearest competitor. This was achieved through
                            improved competitiveness and the lending system
                            delivered by the CLS project played a key part in
                            providing that competitiveness.

                            One of the big problems with agile successes is that
                            they get reported in a relative manner e.g. we
                            implemented (pick a method you like e.g. Scrum) and we
                            produced a four fold improvement. There are lots of
                            such anecdotes. Ken even includes one in the Scrum
                            book. However, none of these results are reported as
                            absolute figures in a normalized manner. The success
                            has a lot to do with the starting position. It's
                            always easy to improve a very poor organization.

                            So I fail to accept that you can assert that adhoc,
                            self-organizing processes can outperform lean
                            processes which still include aspects such as planned
                            versus actual dates. When you can show me the metrics
                            from real projects reported in a normalized fashion,
                            from projects performed in large businesses with large
                            value-added or revenue generation potential then I
                            might start to believe.

                            Cordially,
                            David
                            --
                            David J. Anderson
                            author of "Agile Management for Software Engineering"
                            http://www.agilemanagement.net/

                            --- Marc Hamann <marc@...> wrote:

                            David,

                            My claim is not that your approach doesn't work; there
                            a innumerable
                            systems that have been built using BDUF or even ad hoc
                            methodologies.
                            I'm sure that many of these conform to various sorts
                            of acceptable metrics.

                            However, such practices are not Agile in general or
                            Scrum in
                            particular. Moreover, they work against the goals and
                            values that Agile
                            and Scrum promote, which is counter-productive if you
                            are trying to combine
                            them with Agile practices.

                            My implication that the team should be fired was
                            provocative, but "that
                            won't work here because of our culture/our
                            programmers/our boss" is the
                            most common objection I hear to using Agile
                            techniques. This seems
                            counter-intuitive to me, since I believe that the
                            great virtue of Agile
                            techniques is that they can be used even in adverse
                            circumstances.

                            You don't have to change the culture to start using
                            the techniques; just do
                            your best to implement the practices, and avoid any
                            practices which work
                            against the goals and values you are trying to
                            promote.

                            It's not magic, it's not easy, but if you don't at
                            least try the best you
                            can, you can't claim to be doing Scrum or Agile.

                            Regards,

                            Marc



                            =====
                            David J Anderson
                            Author of "Agile Management for Software Engineering"
                            http://www.agilemanagement.net/

                            __________________________________
                            Do you Yahoo!?
                            The New Yahoo! Shopping - with improved product search
                            http://shopping.yahoo.com
                          • Marc Hamann
                            David, ... I agree that doing agile is not an end in and of itself. Providing return on investment and return on expectation is the real goal. We do this by
                            Message 13 of 20 , Oct 6, 2003
                            • 0 Attachment
                              David,

                              >I think there is a danger of "doing agile" for the
                              >sake of it. The real goal is to make more money by
                              >delivering more value faster to the end of value
                              >chain.

                              I agree that "doing agile" is not an end in and of itself. Providing
                              return on investment and return on expectation is the real goal.

                              We do this by producing working software that fulfills the needs of the
                              customer. I'm quite skeptical that we can do this by producing "metrics".

                              The only metric that counts is how many of the customer's requirements we
                              clear out of the backlog by delivering well-made, working software for a
                              given cost.
                              Any other metric may or may not be a helpful guideline to help you
                              accomplish that primary goal, but some metrics are blatantly
                              counter-productive because they encourage the wrong behaviour.

                              I'm glad you agree that SLOC is not a good metric; crappy code can eat up
                              WAY more lines than good code. ;-)

                              >One of the big problems with agile successes is that
                              >they get reported in a relative manner

                              I suppose part of our disagreement comes from our different concerns. You
                              want to bring a scientific study to the successes of particular cases.
                              Though I'm skeptical that you can ever factor out all the variables that go
                              into a large project and isolate with absolute objectivity that one method
                              is better than another, I can understand why you want to do this.

                              However, I'm in the trenches, and the only reason I give a fig about the
                              Agile methodologies is that they offer approaches and techniques that solve
                              my real day to day challenges in building good, reliable, low cost
                              software. What Chrysler did, or some offshore bank, though perhaps of
                              academic interest will not sway me one way or another as to whether those
                              techniques will work for me. If they work for me, I use them, if they
                              don't, I won't.

                              >So I fail to accept that you can assert that adhoc,
                              >self-organizing processes can outperform lean
                              >processes which still include aspects such as planned
                              >versus actual dates.

                              Well, since we are unlikely to find mutually satisfactory metrics for
                              determining performance, I don't think I would make that claim. ;-)

                              However, I do know that professionals who are too valuable to be fired are
                              smart enough to tell when they are being micro-managed or babied, and
                              neither of these brings out high morale, creativity and "can do" spirit.

                              Remember that people will focus their energies on whatever you measure to
                              evaluate them. If you measure their ability to estimate rather than their
                              ability to produce working software, you will encourage them to pace their
                              work to the plan, rather than doing the best they can always, to avoid
                              taking risks that might throw their estimations off, even if it would
                              improve the product.

                              I can't see how that will improve productivity. In fact, I wouldn't
                              believe it were so even if you showed me some metrics. ;-)

                              >When you can show me the metrics
                              >from real projects reported in a normalized fashion,
                              >from projects performed in large businesses with large
                              >value-added or revenue generation potential then I
                              >might start to believe.

                              When the Buddha was asked "Why should I believe what your saying?", he
                              replied "Don't believe it. Try it for yourself ."

                              I can't improve on that. ;-)

                              Marc
                            • Daniel Gackle
                              Thanks to everyone who replied on estimates vs. actuals. The best thing for me was seeing the diversity of opinion on it, which helped me realize we can just
                              Message 14 of 20 , Oct 10, 2003
                              • 0 Attachment
                                Thanks to everyone who replied on estimates vs. actuals. The best thing for
                                me was seeing the diversity of opinion on it, which helped me realize we can
                                just be pragmatic. Both Jeff's and Ron's reply of "why not give them what
                                they want if it doesn't cost you anything" (my paraphrase) seems applicable
                                to our situation. For me this is part of learning to distinguish the
                                essential stuff from the secondary stuff (which can just flex).

                                I remain perplexed about exactly what people think they're going to get out
                                of such numbers. Maybe my problem is that word "exactly". Maybe they just
                                have a fuzzy idea of what they think they'll get... something vague but
                                comforting, like "more control".

                                Daniel
                              • David J. Anderson
                                ... I remain perplexed about exactly what people think they re going to get out of such numbers. Maybe my problem is that word exactly . Maybe they just have
                                Message 15 of 20 , Oct 11, 2003
                                • 0 Attachment
                                  --- Daniel Gackle <gackle@...> wrote:

                                  I remain perplexed about exactly what people think
                                  they're going to get out
                                  of such numbers. Maybe my problem is that word
                                  "exactly". Maybe they just
                                  have a fuzzy idea of what they think they'll get...
                                  something vague but
                                  comforting, like "more control".

                                  Daniel

                                  -=-=-

                                  Daniel,

                                  There is a real danger that some mangers/directors
                                  will try to use the data for staff evaluation and use
                                  it to filter out targets for layoffs. I worked with a
                                  director at Sprint who tried to do this. I (somewhat)
                                  successfully turned this into athe more benign,
                                  monitoring of our ability to estimate. This made it
                                  more a metric measuring line managers like me, than a
                                  metric measuring my staff. I saw that as part of my
                                  job - taking the heat away from the workforce.

                                  Hence, I think that you are right to be wary. I don't
                                  think that anyone believes they have more control from
                                  metrics like these - other than they believe that they
                                  will get early warning of slippages.

                                  People I work with who believe in short-time window
                                  planned versus actual, i.e. only planned out up to two
                                  weeks ahead, believe that it provides commitment and
                                  focus when that can't be achieved through other means
                                  such as daily standups.

                                  I don't know anyone who genuinely believes that
                                  fine-grained planned versus actual dates works beyond
                                  that time period. Trying to make a full release plan
                                  in advance is just inviting Gantt Chart hell and
                                  providing busy work for project managers anxious to
                                  keep their jobs and their PMI qualifications.
                                  Sometimes, you should look to the personal motivation
                                  and the extremely local optimum for an explanation of
                                  why something is the way it is.

                                  David



                                  =====
                                  David J Anderson
                                  Author of "Agile Management for Software Engineering"
                                  http://www.agilemanagement.net/

                                  __________________________________
                                  Do you Yahoo!?
                                  The New Yahoo! Shopping - with improved product search
                                  http://shopping.yahoo.com
                                • Ken Schwaber
                                  In some instances we re going to be stuck with tracking actuals. Mel Pullen pointed out that this may be a symptom of displaced management trying to figure out
                                  Message 16 of 20 , Oct 12, 2003
                                  • 0 Attachment
                                    In some instances we're going to be stuck with tracking actuals. Mel Pullen
                                    pointed out that this may be a symptom of displaced management trying to
                                    figure out how they are adding value, and relying on what they have used in
                                    the past to add that value. They have always tried to improve predictabity
                                    in the past, let's try to do it in the future.

                                    I've always tried to get management to do another job. I've tried to get
                                    them to see how well or badly a team does, what it can produce Sprint by
                                    Sprint. Then I ask them to actually do the job of management - figure out
                                    what to do based on what the team has been able to build. Should release
                                    dates be changed? Should the project be decommissioned? Should functionality
                                    be dropped from this release? Should the team's expertise be improved in
                                    some areas? To me, these are all reasonable areas of management expertise,
                                    not throwing the problem back on the team by saying, "improve your
                                    estimates."

                                    Ken

                                    -----Original Message-----
                                    From: Mike Cohn [mailto:mike@...]
                                    Sent: Sunday, October 05, 2003 9:57 AM
                                    To: scrumdevelopment@yahoogroups.com
                                    Subject: RE: [scrumdevelopment] What's wrong with tracking estimates vs.
                                    actuals?


                                    I'd say you shouldn't do it because it doesn't add value commensurate with
                                    its cost. Don't argue with your bosses that it "adds no value" because
                                    comparing what you originally thought a task would take with what it did
                                    take can help make you a better estimator.

                                    But, it can be time-consuming to track actuals, especially for a full team
                                    where some on the team are probably already decent estimators.

                                    Because Scrum already has solid mechanisms for monitoring whether all the
                                    work gets done in a sprint (high team commitment, daily burndown charts,
                                    daily scrum, and so on), Scrum does not have the same reliance on early and
                                    accurating estimating that a predictive or waterfall approach does.

                                    So--the cost to gather actuals is the same in Scrum or waterfall. The
                                    benefit in Scrum is greatly reduced.

                                    --Mike



                                    > -----Original Message-----
                                    > From: Daniel Gackle [mailto:gackle@...]
                                    > Sent: Sunday, October 05, 2003 12:35 AM
                                    > To: scrumdevelopment@yahoogroups.com
                                    > Subject: [scrumdevelopment] What's wrong with tracking estimates vs.
                                    > actuals?
                                    >
                                    > At my company, some managers believe in tracking estimates vs. actuals.
                                    > They
                                    > like what my team is doing with Scrum, but they'd like it better if we
                                    > gave
                                    > them a spreadsheet every month matching the estimate for each task with
                                    > the
                                    > hours actually spent on that task.
                                    >
                                    > I feel bad about this and don't want to do it. It feels contrary to the
                                    > spirit of self-organizing teams, like someone is looking over our
                                    > shoulder.
                                    > Yet I can't fully articulate what's wrong with it.
                                    >
                                    > Can anyone help me get clearer on this?
                                    >
                                    > - Daniel
                                    >
                                    > -----Original Message-----
                                    > From: "Gamble, Ken" <ken.gamble@...>
                                    > Subject: New Scrum Article Available
                                    >
                                    > No matter how well someone measures past estimates against actuals even a
                                    > small change in the estimate can have a big effect on the outcome of a
                                    > chaotic/complex process no matter how good the model is or the resolution
                                    > of
                                    > the measurements.
                                    >
                                    > What this means for software development is that even though we can use
                                    > previous estimates as part of a process model for delivering software we
                                    > have to keep a sharp eye (Scrum management) on the process because it can
                                    > wonder off course simply because of these small values, even if the
                                    > process
                                    > model is very tightly defined.
                                    >
                                    >
                                    >
                                    > To Post a message, send it to: scrumdevelopment@...
                                    > To Unsubscribe, send a blank message to: scrumdevelopment-
                                    > unsubscribe@...
                                    >
                                    > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/




                                    To Post a message, send it to: scrumdevelopment@...
                                    To Unsubscribe, send a blank message to:
                                    scrumdevelopment-unsubscribe@...

                                    Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                                  • Steven Gordon
                                    I have worked on many projects in the past where the elimination of labor was considered a significant benefit, if not the greatest benefit of the resulting
                                    Message 17 of 20 , Oct 12, 2003
                                    • 0 Attachment
                                      I have worked on many projects in the past where the elimination of labor was considered a significant benefit, if not the greatest benefit of the resulting process and/or product.

                                      Why would the elimination of unnecessary management labor not be a benefit to a client? Maybe, the case needs to be made further up the food chain?


                                      -----Original Message-----
                                      From: Ken Schwaber [mailto:ken.schwaber@...]
                                      Sent: Sun 10/12/2003 4:03 AM
                                      To: scrumdevelopment@yahoogroups.com
                                      Cc:
                                      Subject: RE: [scrumdevelopment] What's wrong with tracking estimates vs. actuals?

                                      In some instances we're going to be stuck with tracking actuals. Mel Pullen
                                      pointed out that this may be a symptom of displaced management trying to
                                      figure out how they are adding value, and relying on what they have used in
                                      the past to add that value. They have always tried to improve predictabity
                                      in the past, let's try to do it in the future.

                                      I've always tried to get management to do another job. I've tried to get
                                      them to see how well or badly a team does, what it can produce Sprint by
                                      Sprint. Then I ask them to actually do the job of management - figure out
                                      what to do based on what the team has been able to build. Should release
                                      dates be changed? Should the project be decommissioned? Should functionality
                                      be dropped from this release? Should the team's expertise be improved in
                                      some areas? To me, these are all reasonable areas of management expertise,
                                      not throwing the problem back on the team by saying, "improve your
                                      estimates."

                                      Ken

                                      -----Original Message-----
                                      From: Mike Cohn [mailto:mike@...]
                                      Sent: Sunday, October 05, 2003 9:57 AM
                                      To: scrumdevelopment@yahoogroups.com
                                      Subject: RE: [scrumdevelopment] What's wrong with tracking estimates vs.
                                      actuals?


                                      I'd say you shouldn't do it because it doesn't add value commensurate with
                                      its cost. Don't argue with your bosses that it "adds no value" because
                                      comparing what you originally thought a task would take with what it did
                                      take can help make you a better estimator.

                                      But, it can be time-consuming to track actuals, especially for a full team
                                      where some on the team are probably already decent estimators.

                                      Because Scrum already has solid mechanisms for monitoring whether all the
                                      work gets done in a sprint (high team commitment, daily burndown charts,
                                      daily scrum, and so on), Scrum does not have the same reliance on early and
                                      accurating estimating that a predictive or waterfall approach does.

                                      So--the cost to gather actuals is the same in Scrum or waterfall. The
                                      benefit in Scrum is greatly reduced.

                                      --Mike



                                      > -----Original Message-----
                                      > From: Daniel Gackle [mailto:gackle@...]
                                      > Sent: Sunday, October 05, 2003 12:35 AM
                                      > To: scrumdevelopment@yahoogroups.com
                                      > Subject: [scrumdevelopment] What's wrong with tracking estimates vs.
                                      > actuals?
                                      >
                                      > At my company, some managers believe in tracking estimates vs. actuals.
                                      > They
                                      > like what my team is doing with Scrum, but they'd like it better if we
                                      > gave
                                      > them a spreadsheet every month matching the estimate for each task with
                                      > the
                                      > hours actually spent on that task.
                                      >
                                      > I feel bad about this and don't want to do it. It feels contrary to the
                                      > spirit of self-organizing teams, like someone is looking over our
                                      > shoulder.
                                      > Yet I can't fully articulate what's wrong with it.
                                      >
                                      > Can anyone help me get clearer on this?
                                      >
                                      > - Daniel
                                      >
                                      > -----Original Message-----
                                      > From: "Gamble, Ken" <ken.gamble@...>
                                      > Subject: New Scrum Article Available
                                      >
                                      > No matter how well someone measures past estimates against actuals even a
                                      > small change in the estimate can have a big effect on the outcome of a
                                      > chaotic/complex process no matter how good the model is or the resolution
                                      > of
                                      > the measurements.
                                      >
                                      > What this means for software development is that even though we can use
                                      > previous estimates as part of a process model for delivering software we
                                      > have to keep a sharp eye (Scrum management) on the process because it can
                                      > wonder off course simply because of these small values, even if the
                                      > process
                                      > model is very tightly defined.
                                      >
                                      >
                                      >
                                      > To Post a message, send it to: scrumdevelopment@...
                                      > To Unsubscribe, send a blank message to: scrumdevelopment-
                                      > unsubscribe@...
                                      >
                                      > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/




                                      To Post a message, send it to: scrumdevelopment@...
                                      To Unsubscribe, send a blank message to:
                                      scrumdevelopment-unsubscribe@...

                                      Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/





                                      Yahoo! Groups Sponsor
                                      ADVERTISEMENT
                                      Click Here! <http://rd.yahoo.com/M=244522.3707890.4968055.1261774/D=egroupweb/S=1707209021:HM/A=1595054/R=0/SIG=124ukap9t/*http://ashnin.com/clk/muryutaitakenattogyo?YH=3707890&yhad=1595054>
                                      <http://us.adserver.yahoo.com/l?M=244522.3707890.4968055.1261774/D=egroupmail/S=:HM/A=1595054/rand=270339561>

                                      To Post a message, send it to: scrumdevelopment@...
                                      To Unsubscribe, send a blank message to: scrumdevelopment-unsubscribe@...

                                      Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service <http://docs.yahoo.com/info/terms/> .
                                    • Edmund Schweppe
                                      ... What will the client do with the managers whose unnecessary management labor will be eliminated? Or, more to the point, what do the affected managers
                                      Message 18 of 20 , Oct 12, 2003
                                      • 0 Attachment
                                        Steven Gordon wrote:
                                        > I have worked on many projects in the past where the
                                        > elimination of labor was considered a significant benefit,
                                        > if not the greatest benefit of the resulting process and/or
                                        > product.
                                        >
                                        > Why would the elimination of unnecessary management labor not
                                        > be a benefit to a client? Maybe, the case needs to be made
                                        > further up the food chain?

                                        What will the client do with the managers whose "unnecessary management
                                        labor" will be eliminated? Or, more to the point, what do the affected
                                        managers *think* the client will do with them?

                                        If the particular client has a history of dealing with "unnecessary"
                                        positions by firing the persons therein, the affected managers are going
                                        to be *very* strongly motivated to make sure *their* positions are seen
                                        as "necessary".

                                        --
                                        Edmund Schweppe -- schweppe@... -- http://schweppe.home.tiac.net
                                        The opinions expressed herein are at best coincidentally related to
                                        those of any past, present or future employer.
                                      • Steven Gordon
                                        The main difference is whose position is being eliminated. It bothers me that projects which reduce how much labor it takes to do production
                                        Message 19 of 20 , Oct 12, 2003
                                        • 0 Attachment
                                          The main difference is whose position is being eliminated.

                                          <Rant mode on>

                                          It bothers me that projects which reduce how much labor it takes to do production will lead to reductions in force, but projects that reduce how much management is required just leads to managers with time on their hands.

                                          And managers with time on their hands leads to micromanagement activities like tracking estimates vs. actuals in a process where the metric is not highly correlated to success. Next, they will try to weigh the process down with collecting even more metrics that are not success factors, and start applying 6-sigma optimizations on them. Will anybody notice that these measured metrics will improve, but actual productivity and true successes will decrease? Probably not, because they will define productivity and success in terms of the metrics they are collecting and analyzing.

                                          <Rant mode off>

                                          Sorry if I offended any managers here.

                                          -----Original Message-----
                                          From: Edmund Schweppe [mailto:schweppe@...]
                                          Sent: Sun 10/12/2003 10:21 AM
                                          To: scrumdevelopment@yahoogroups.com
                                          Cc:
                                          Subject: Re: [scrumdevelopment] What's wrong with tracking estimates vs. actuals?

                                          Steven Gordon wrote:
                                          > I have worked on many projects in the past where the
                                          > elimination of labor was considered a significant benefit,
                                          > if not the greatest benefit of the resulting process and/or
                                          > product.
                                          >
                                          > Why would the elimination of unnecessary management labor not
                                          > be a benefit to a client? Maybe, the case needs to be made
                                          > further up the food chain?

                                          What will the client do with the managers whose "unnecessary management
                                          labor" will be eliminated? Or, more to the point, what do the affected
                                          managers *think* the client will do with them?

                                          If the particular client has a history of dealing with "unnecessary"
                                          positions by firing the persons therein, the affected managers are going
                                          to be *very* strongly motivated to make sure *their* positions are seen
                                          as "necessary".

                                          --
                                          Edmund Schweppe -- schweppe@... -- http://schweppe.home.tiac.net
                                          The opinions expressed herein are at best coincidentally related to
                                          those of any past, present or future employer.



                                          Yahoo! Groups Sponsor
                                          <http://rd.yahoo.com/M=259395.3614674.4902533.1261774/D=egroupweb/S=1707209021:HM/A=1524963/R=0/SIG=12o885gmo/*http://hits.411web.com/cgi-bin/autoredir?camp=556&lineid=3614674?=egroupweb&pos=HM>
                                          <http://us.adserver.yahoo.com/l?M=259395.3614674.4902533.1261774/D=egroupmail/S=:HM/A=1524963/rand=240962557>

                                          To Post a message, send it to: scrumdevelopment@...
                                          To Unsubscribe, send a blank message to: scrumdevelopment-unsubscribe@...

                                          Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service <http://docs.yahoo.com/info/terms/> .
                                        • David J. Anderson
                                          Steve, You are defining what bad managers do - managers who don t know what management is and probably have never been trained as proper managers. For the
                                          Message 20 of 20 , Oct 12, 2003
                                          • 0 Attachment
                                            Steve,

                                            You are defining what bad managers do - managers who
                                            don't know what management "is" and probably have
                                            never been trained as proper managers.

                                            For the rest of us good managers - we just tune you
                                            out :-)

                                            David


                                            --- Steven Gordon <sagordon@...> wrote:

                                            The main difference is whose position is being
                                            eliminated.

                                            <Rant mode on>

                                            It bothers me that projects which reduce how much
                                            labor it takes to do production will lead to
                                            reductions in force, but projects that reduce how much
                                            management is required just leads to managers with
                                            time on their hands.

                                            And managers with time on their hands leads to
                                            micromanagement activities like tracking estimates vs.
                                            actuals in a process where the metric is not highly
                                            correlated to success. Next, they will try to weigh
                                            the process down with collecting even more metrics
                                            that are not success factors, and start applying
                                            6-sigma optimizations on them. Will anybody notice
                                            that these measured metrics will improve, but actual
                                            productivity and true successes will decrease?
                                            Probably not, because they will define productivity
                                            and success in terms of the metrics they are
                                            collecting and analyzing.

                                            <Rant mode off>

                                            Sorry if I offended any managers here.



                                            __________________________________
                                            Do you Yahoo!?
                                            The New Yahoo! Shopping - with improved product search
                                            http://shopping.yahoo.com
                                          Your message has been successfully submitted and would be delivered to recipients shortly.