Re: Measuring Planning Performance
Sorry I'm late responding.
To understand what I mean by measuring planning performance let's step back to consider what I mean by planning. I contend planning is a conversation for producing coherency of commitments among team members to deliver on the promises of the project. The project promises are the big 'deliverables,' usually to the customer, but also to interested third parties. We engage in conversations with people who assign work and or perform work in an effort to coordinate with each other. As some people have noted, when we fail to coordinate well we incur delays and produce waste. For instance, if I say I'm working on a block of code 'til Thursday at which time I plan to check it in, someone else may mobilize to check the code out for their own work on Friday. In some organizations there isn't even this level of conversation...I don't tell others what I'l working on let alone when I plan to make it available for others. 'Signalling and committing,' as Mary puts it, is the everyday-always planning activity. We do this planning to stay in sync with each other. It is this planning in particular that we want to get good at.
Planning (system) performance can be measured in a number of ways. I suggest two for starters. First, are we able to present work to team members that is in the desired sequence and in a condition to be started and finished? Second, are people taking assignments that they can complete as they say they will complete? Many people think of the first measure as an indicator of management performance. However, we might be running the project autonomously. In that case it would be the measure of our collective planning and make-ready actions. Performance is calculated as 'ready work'/work intended to start.
The second measure is the percent of planned work completed (PPC). It is an indication of how well people are matched to assignments and their abililty to work as they plan to work. Oftentimes the work is in a ready state, but the 'wrong' person is put on the task. Other times people may be well-matched for the task, but they find themselves pulled away from the work they planned to do. Either way, to the extent that the task completion releases work for others, the project is delayed when someone doesn't complete as they say they will. Performance is calculated as tasks completed/tasks planned for completion. No 'credit' is given for completing tasks not planned for completion. This is a measure of planning performance not productivity. Also, no credit is given for partial completion. For instance, finishing 90% each on 10 tasks is 0% complete not 90%. The concern we have is for the release of work. Task completions release work for others.
We have quite a bit of data on planning system performance. What I find most encouraging is on projects where PPC < 50% costs run 115% of budget; however where PPC > 70% costs run 85% of budget. In the first case half of all work is not completed as people say they will complete it leaving others waiting on them and project managers spending their time expediting. In the second case less than 30% of the task completions are delayed. There's far less waiting on others and expediting. The difference is noticeable in day-to-day activities, in the mood of the team members, and in the quality of the output. BTW, the usual 'well-run' conventional project's PPC < 50%. This is stark evidence that doing better at the wrong thing will not get you good results. We must change the project practices if we want better performance.
You can take this measurement thing farther. For instance, I ask people to do a 'five-why' analysis of every plan failure (where they don't complete as they said they would). I use standard reasons for variance, collect the data over time, and then do problem-cause removal to eliminate the source of the plan failure. Surprisingly, the vast majority of plan failures are in the control of the project participants. In other words, while people readily agree that "shit happens" is why we don't carry out projects as we intend, that is wrong. We are our own sources of variability.
I hope this helps.
p.s. My friend Joe, Joe's Lean Diary, is in the midst of making 5,000 apple pies with his son's school. They do this each year as a fund-raiser. He decided to change over from a 'batch' mentality to 'single-piece flow'. It entailed a project to do the changeover. The promise is the 5,000 pies in two days. After one day he got about 2,000 pies. Today is the second day. Check-in on his weblog to see how he did.
Date: Tue, 15 Oct 2002 11:59:05 -0400
From: "Ken Schwaber"
Subject: RE: Re: Planning vs. Predicting
"We can begin measuring the performance of our planning." That's a pretty
significant shift from measuring why people can't do as we plan! I really
like the human/people orientation of your comments, which I believe is the
basis of agile processes. Do what is possible, do the best you can ... not,
"do what we tell you to do."
I think that the more complicated and stressful life becoms, the more people
want to simplify and retreat. Working through complexities and anomalies
takes time and effort, and communication with the most complex thing
around - other people. So the natural desire is to believe that the plan can
be enforced. I think the major benefit of agile is that it "dumbs down" the
whole process so we stop expecting comlicated mechanisms and plans to
substitute for communication, observation, and collaboration.
Get a daily dose of the changes underway in project management at Reforming Project Management. What's a weblog? http://weblog.halmacomber.com