Loading ...
Sorry, an error occurred while loading the content.

RE: [CMMi Process Improvement] ML4 Glitches

Expand Messages
  • Neeru Walia
    Hi All, Can somebody please help me on the questions mentioned below? Thanks, Neeru From: cmmi_process_improvement@yahoogroups.com
    Message 1 of 15 , Sep 8, 2010
    View Source
    • 0 Attachment

       

      Hi All,

       

      Can somebody please help me on the questions mentioned below?

       

      Thanks,

      Neeru

       

      From: cmmi_process_improvement@yahoogroups.com [mailto:cmmi_process_improvement@yahoogroups.com] On Behalf Of Neeru Walia
      Sent: Wednesday, August 25, 2010 3:22 PM
      To: cmmi_process_improvement@yahoogroups.com
      Subject: [CMMi Process Improvement] ML4 Glitches

       

       

      Hi All,

       

      This is first time I’m writing to this group and hope I will get expert’s feedbackJ

       

      Our organization is going for CMMI ML 4 by next year  and by this time we have all the PPMs and PPBs in place. But I have two question here:

       

      1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

       

       

      2.       How often Project should run the simulation to validate the success results are within thresholds or not?

       

       

      3.       When a new project has started it will use the Org. PCB and PPM (for similar nature project) to run the simulation and achieve the probability of success? Say if project has its own data  points  (ex: 15 releases data) then the new project should use their own data(Mean, Stdev) while running simulation or they should add their data point into Org PCB and the new mean and stdev should be used  run the simulation?  

       

       

       Hope, I have made my question clear.

       

      Thanks,

      Neeru

    • praveen
      Hi   I am not an expert but let me try to answer first 2 questions from my CMMI ML5 implementation experience:   1.       How often organization should
      Message 2 of 15 , Sep 9, 2010
      View Source
      • 0 Attachment
        Hi
         
        I am not an expert but let me try to answer first 2 questions from my CMMI ML5 implementation experience:
         

        1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

         

        Ans.1. PCB can be revised at an interval of 4 - 6 months. If there is changes in capability, its always good to revise the PPM alsp.

         

         

        2.       How often Project should run the simulation to validate the success results are within thresholds or not?

         

        Ans2. Project can run the simulation at end of phase / milestone



        Thanks & Regards

        Praveen Harkawat
        L & T Mumbai India

        --- On Thu, 9/9/10, Neeru Walia <neeru.walia@...> wrote:

        From: Neeru Walia <neeru.walia@...>
        Subject: RE: [CMMi Process Improvement] ML4 Glitches
        To: cmmi_process_improvement@yahoogroups.com
        Date: Thursday, September 9, 2010, 10:27 AM

         

         

        Hi All,

         

        Can somebody please help me on the questions mentioned below?

         

        Thanks,

        Neeru

         

        From: cmmi_process_improvement@yahoogroups.com [mailto:cmmi_process_improvement@yahoogroups.com] On Behalf Of Neeru Walia
        Sent: Wednesday, August 25, 2010 3:22 PM
        To: cmmi_process_improvement@yahoogroups.com
        Subject: [CMMi Process Improvement] ML4 Glitches

         

         

        Hi All,

         

        This is first time I’m writing to this group and hope I will get expert’s feedbackJ

         

        Our organization is going for CMMI ML 4 by next year  and by this time we have all the PPMs and PPBs in place. But I have two question here:

         

        1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

         

         

        2.       How often Project should run the simulation to validate the success results are within thresholds or not?

         

         

        3.       When a new project has started it will use the Org. PCB and PPM (for similar nature project) to run the simulation and achieve the probability of success? Say if project has its own data  points  (ex: 15 releases data) then the new project should use their own data(Mean, Stdev) while running simulation or they should add their data point into Org PCB and the new mean and stdev should be used  run the simulation?  

         

         

         Hope, I have made my question clear.

         

        Thanks,

        Neeru


      • Neeru Walia
        Thank you so much Pat for reply and the courtesy to forward me this email again (which somehow I have lost )J All your responses are satisfactory and helped
        Message 3 of 15 , Sep 10, 2010
        View Source
        • 0 Attachment

          Thank you so much Pat for reply and the courtesy to forward me this email again (which somehow I have lost )J

           

          All your responses are satisfactory and helped me.

           

          Thanks,

          Neeru

           

           

           

          Sent: Wednesday, August 25, 2010 4:51 AM

          Subject: [CMMi Process Improvement] ML4 Glitches

           

           

          Our organization is going for CMMI ML 4 by next year  and by this time we have all the PPMs and PPBs in place. But I have two question here:

          1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

          You want to use the incorporate the best available information into your PPBs and PPMs - that which gives you the best insight into the likely outcomes for the next project.  Although there is no "standard period" for updating these things, I do see a relatively broad set of updated practices out there.  On the one extreme, I've seen organizations that only update them once a year (which seems to violate my first statement!)  On the other hand, I've seen organizations that update them with each execution of the process - especially when they have a small number of data points on which to base their PPBs and PPM.  Most commonly, I see updates made on a quarterly basis.

          2.       How often Project should run the simulation to validate the success results are within thresholds or not?

          How often do you run a critical path analysis?  Or your earned value numbers?  How often do you look at your schedule and budget?  PPMs and PPBs are simply tools that the project manager uses to gain insight into future events.  One of my favorite statements made during an ML5 appraisal was, "You'll know that you have a really good Process Performance Model when it predicts events that don't occur!"  The point that was being made is that the PPM projects likely outcomes.  If you don't LIKE those likely outcomes, you will take steps now to experience an alternative future.  THAT's how they should be used.  So, if you had a crystal ball that could forecast your future, how often would YOU gaze into it??? 

          3.       When a new project has started it will use the Org. PCB and PPM (for similar nature project) to run the simulation and achieve the probability of success? Say if project has its own data  points  (ex: 15 releases data) then the new project should use their own data(Mean, Stdev) while running simulation or they should add their data point into Org PCB and the new mean and stdev should be used  run the simulation?  

          If a project has sufficient data to calibrate the PPM, then that is probably the best data to use.  The same holds true for process performance baselines.  HOWEVER, why not use both?  Use the project data as the primary predictor and the organizational data as a sanity check?  Many organizations do this with estimation at ML3.  The project estimates size, effort, cost, and schedule based on their own historical data, and then goes to the organizational measurement repository to find similar projects against which they can perform a sanity check.

          Most PPMs just don't take that long to execute - so execute them early and often, and with whatever data can be used to provide robust insight into likely outcomes. 

           

           Hope, I have made my question clear.

          Thanks,

          Neeru

        • EDWARD F WELLER III
          In a presentation Will Hayes of the SEI made some time ago he commented (paraphrasing) if you update your PPBs every quarter, you do not understand High
          Message 4 of 15 , Sep 10, 2010
          View Source
          • 0 Attachment
            
            In a presentation Will Hayes of the SEI made some time ago he commented (paraphrasing) "if you update your PPBs every quarter, you do not understand High Maturity"
             
            PPBs are updated when
            1) there is a statistically significant difference in before/after performance where there is a process change in the before/after
            2) For those processes with trends, when one of the stability tests indicate process performance has reached a new area of capability
            in the case of trends, one could also apply the technique of Burr and Owen - use the slope of the regression to calculate the control limits
             
            yes - you want to evaluate periodically, but change the PCBs for reason only, not just to do a recalculation
             
            See also Pat O'Toole's comments on when to run simulation models - essentially run them when there is a reason (event driven) or periodically (you need to figure out the cost benefit for how often)
            ----- Original Message -----
             
            Ed
            From: praveen
            Sent: Thursday, September 09, 2010 8:34 PM
            Subject: RE: [CMMi Process Improvement] ML4 Glitches

             

            Hi
             
            I am not an expert but let me try to answer first 2 questions from my CMMI ML5 implementation experience:
             

            1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

            Ans.1. PCB can be revised at an interval of 4 - 6 months. If there is changes in capability, its always good to revise the PPM alsp.

            2.       How often Project should run the simulation to validate the success results are within thresholds or not?

            Ans2. Project can run the simulation at end of phase / milestone



            Thanks & Regards

            Praveen Harkawat
            L & T Mumbai India

            --- On Thu, 9/9/10, Neeru Walia <neeru.walia@...> wrote:

            From: Neeru Walia <neeru.walia@...>
            Subject: RE: [CMMi Process Improvement] ML4 Glitches
            To: cmmi_process_improvement@yahoogroups.com
            Date: Thursday, September 9, 2010, 10:27 AM

             

             

            Hi All,

             

            Can somebody please help me on the questions mentioned below?

            Thanks,

            Neeru

            From: cmmi_process_improvement@yahoogroups.com [mailto:cmmi_process_improvement@yahoogroups.com] On Behalf Of Neeru Walia
            Sent: Wednesday, August 25, 2010 3:22 PM
            To: cmmi_process_improvement@yahoogroups.com
            Subject: [CMMi Process Improvement] ML4 Glitches

            Hi All,

             

            This is first time I’m writing to this group and hope I will get expert’s feedbackJ

             

            Our organization is going for CMMI ML 4 by next year  and by this time we have all the PPMs and PPBs in place. But I have two question here:

            1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

            2.       How often Project should run the simulation to validate the success results are within thresholds or not?

            3.       When a new project has started it will use the Org. PCB and PPM (for similar nature project) to run the simulation and achieve the probability of success? Say if project has its own data  points  (ex: 15 releases data) then the new project should use their own data(Mean, Stdev) while running simulation or they should add their data point into Org PCB and the new mean and stdev should be used  run the simulation?  

             Hope, I have made my question clear.

            Thanks,

            Neeru


          • Pat OToole
            Ed and I have swapped some private email over this one (primarily based on my previous post), and I think we concluded that our difference of opinion revolves
            Message 5 of 15 , Sep 10, 2010
            View Source
            • 0 Attachment
               
              Ed and I have swapped some private email over this one (primarily based on my previous post), and I think we concluded that our difference of opinion revolves around the meaning of the word "update." 
               
              If by "update" is meant "establishing a new baseline," then I wholeheartedly agree with Ed - you do that when there is a process/tool change OR the process performance data indicates the process performance is significantly different than the previous baseline.  This latter motivation could represent improved performance - based on enhanced experience/skill related to process performance or other factors; or it could represent a degradation of performance - based on a change in personnel, erosion of process discipline, etc.  In any event, this would represent a new level of performance that warrants establishment of a new process performance baseline (and changes to associated process performance models).
               
              On the other hand, if by "update" is meant incorporating additional data into the data pool used to establish the process performance baseline, then I would argue that this should be done much more aggressively.  Process performance baselines that are established based on 20 data points are likely to have significantly wider control limits than those based on 100 data points.  If you continue to rely on your 20-point PPB and its wider control limits, you may inadvertently miss signals sent by data from current process executions.
               
              Also, one should consider whether or not the "learning curve" has been overcome prior to the baseline being established.  In other words, if the "formal inspection" process has only recently been introduced, one would expect broader fluctuation in the resulting data than if the formal inspection process was deeply institutionalized.  In the case of recently introduced processes, it would not be surprising to find that the center line drifts in a positive direction as deeper process institutionalization occurs - and the control limits also migrate inward as process variation is reduced.  In other words, until process performance is statistically stable, you really aren't in a position to employ statistical techniques that rely on statistical stability as a prerequisite!  (Oh yeah, you can still generate a control chart, but knowing that your coding process yields 0 - 50 defects per KSLOC is not all that helpful!)
               
              Hope this helps,
               
              Pat
               
               
               
               

              Sent: Friday, September 10, 2010 10:26 AM
              Subject: Re: [CMMi Process Improvement] ML4 Glitches

               

              

              In a presentation Will Hayes of the SEI made some time ago he commented (paraphrasing) "if you update your PPBs every quarter, you do not understand High Maturity"
               
              PPBs are updated when
              1) there is a statistically significant difference in before/after performance where there is a process change in the before/after
              2) For those processes with trends, when one of the stability tests indicate process performance has reached a new area of capability
              in the case of trends, one could also apply the technique of Burr and Owen - use the slope of the regression to calculate the control limits
               
              yes - you want to evaluate periodically, but change the PCBs for reason only, not just to do a recalculation
               
              See also Pat O'Toole's comments on when to run simulation models - essentially run them when there is a reason (event driven) or periodically (you need to figure out the cost benefit for how often)
              ----- Original Message -----
               
              Ed
              From: praveen
              Sent: Thursday, September 09, 2010 8:34 PM
              Subject: RE: [CMMi Process Improvement] ML4 Glitches

               

              Hi
               
              I am not an expert but let me try to answer first 2 questions from my CMMI ML5 implementation experience:
               

              1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

              Ans.1. PCB can be revised at an interval of 4 - 6 months. If there is changes in capability, its always good to revise the PPM alsp.

              2.       How often Project should run the simulation to validate the success results are within thresholds or not?

              Ans2. Project can run the simulation at end of phase / milestone



              Thanks & Regards

              Praveen Harkawat
              L & T Mumbai India

              --- On Thu, 9/9/10, Neeru Walia <neeru.walia@...> wrote:

              From: Neeru Walia <neeru.walia@...>
              Subject: RE: [CMMi Process Improvement] ML4 Glitches
              To: cmmi_process_improvement@yahoogroups.com
              Date: Thursday, September 9, 2010, 10:27 AM

               

               

              Hi All,

               

              Can somebody please help me on the questions mentioned below?

              Thanks,

              Neeru

              From: cmmi_process_improvement@yahoogroups.com [mailto:cmmi_process_improvement@yahoogroups.com] On Behalf Of Neeru Walia
              Sent: Wednesday, August 25, 2010 3:22 PM
              To: cmmi_process_improvement@yahoogroups.com
              Subject: [CMMi Process Improvement] ML4 Glitches

              Hi All,

               

              This is first time I’m writing to this group and hope I will get expert’s feedbackJ

               

              Our organization is going for CMMI ML 4 by next year  and by this time we have all the PPMs and PPBs in place. But I have two question here:

              1.       How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?

              2.       How often Project should run the simulation to validate the success results are within thresholds or not?

              3.       When a new project has started it will use the Org. PCB and PPM (for similar nature project) to run the simulation and achieve the probability of success? Say if project has its own data  points  (ex: 15 releases data) then the new project should use their own data(Mean, Stdev) while running simulation or they should add their data point into Org PCB and the new mean and stdev should be used  run the simulation?  

               Hope, I have made my question clear.

              Thanks,

              Neeru


            • efwelleraol
              I think this discussion clearly indicated the arithmetic is far simpler than the understanding of what the squiggly lines mean and how to use them. To really
              Message 6 of 15 , Sep 11, 2010
              View Source
              • 0 Attachment
                I think this discussion clearly indicated the arithmetic is far simpler than the understanding of what the squiggly lines mean and how to use them. To really understand what the data means is to know the organization, know the people, and know the process as it is being executed.
                I remember being shown a chart and calculations showing a statistically significant shift in mean and control limits. When I looked at the data, the first question I asked was "Why are these nn data points identical?" Turns out the values were equal to a full day of effort (8 is an easy number to spot!) The improved process/work had significantly reduced the tail end distribution (thus the shift) but they had completely overlooked the suspicious data. In this case it did not negate the improvement, but any further use of this process for analysis of improvement would require the data recording errors to be fixed.

                Moral of the story: Always look at the raw data

                BTW, I suspect if the presentation had not been in 24 pt powerpoint, the discussion might have been "If the ONLY reason you update your charts is the end of quarter". You do need to continuously evaluate the data. It is actually quite unlikely to have a string of data, even from inspections, that will give you 30+ data points from the same subgroup. Team members may change, the "module" may change (ease of review, complexity, producer, etc.)

                Ed

                --- In cmmi_process_improvement@yahoogroups.com, "Pat OToole" <PACT.otoole@...> wrote:
                >
                >
                > Ed and I have swapped some private email over this one (primarily based on my previous post), and I think we concluded that our difference of opinion revolves around the meaning of the word "update."
                >
                > If by "update" is meant "establishing a new baseline," then I wholeheartedly agree with Ed - you do that when there is a process/tool change OR the process performance data indicates the process performance is significantly different than the previous baseline. This latter motivation could represent improved performance - based on enhanced experience/skill related to process performance or other factors; or it could represent a degradation of performance - based on a change in personnel, erosion of process discipline, etc. In any event, this would represent a new level of performance that warrants establishment of a new process performance baseline (and changes to associated process performance models).
                >
                > On the other hand, if by "update" is meant incorporating additional data into the data pool used to establish the process performance baseline, then I would argue that this should be done much more aggressively. Process performance baselines that are established based on 20 data points are likely to have significantly wider control limits than those based on 100 data points. If you continue to rely on your 20-point PPB and its wider control limits, you may inadvertently miss signals sent by data from current process executions.
                >
                > Also, one should consider whether or not the "learning curve" has been overcome prior to the baseline being established. In other words, if the "formal inspection" process has only recently been introduced, one would expect broader fluctuation in the resulting data than if the formal inspection process was deeply institutionalized. In the case of recently introduced processes, it would not be surprising to find that the center line drifts in a positive direction as deeper process institutionalization occurs - and the control limits also migrate inward as process variation is reduced. In other words, until process performance is statistically stable, you really aren't in a position to employ statistical techniques that rely on statistical stability as a prerequisite! (Oh yeah, you can still generate a control chart, but knowing that your coding process yields 0 - 50 defects per KSLOC is not all that helpful!)
                >
                > Hope this helps,
                >
                > Pat
                >
                >
                >
                >
                >
                >
                > From: EDWARD F WELLER III
                > Sent: Friday, September 10, 2010 10:26 AM
                > To: cmmi_process_improvement@yahoogroups.com
                > Subject: Re: [CMMi Process Improvement] ML4 Glitches
                >
                >
                >
                > 
                >
                > In a presentation Will Hayes of the SEI made some time ago he commented (paraphrasing) "if you update your PPBs every quarter, you do not understand High Maturity"
                >
                > PPBs are updated when
                > 1) there is a statistically significant difference in before/after performance where there is a process change in the before/after
                > 2) For those processes with trends, when one of the stability tests indicate process performance has reached a new area of capability
                > in the case of trends, one could also apply the technique of Burr and Owen - use the slope of the regression to calculate the control limits
                >
                > yes - you want to evaluate periodically, but change the PCBs for reason only, not just to do a recalculation
                >
                > See also Pat O'Toole's comments on when to run simulation models - essentially run them when there is a reason (event driven) or periodically (you need to figure out the cost benefit for how often)
                > ----- Original Message -----
                >
                > Ed
                > From: praveen
                > To: cmmi_process_improvement@yahoogroups.com
                > Sent: Thursday, September 09, 2010 8:34 PM
                > Subject: RE: [CMMi Process Improvement] ML4 Glitches
                >
                >
                >
                > Hi
                >
                > I am not an expert but let me try to answer first 2 questions from my CMMI ML5 implementation experience:
                >
                > 1. How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?
                >
                >
                > Ans.1. PCB can be revised at an interval of 4 - 6 months. If there is changes in capability, its always good to revise the PPM alsp.
                >
                >
                >
                > 2. How often Project should run the simulation to validate the success results are within thresholds or not?
                >
                >
                > Ans2. Project can run the simulation at end of phase / milestone
                >
                >
                >
                > Thanks & Regards
                >
                > Praveen Harkawat
                > L & T Mumbai India
                >
                > --- On Thu, 9/9/10, Neeru Walia <neeru.walia@...> wrote:
                >
                >
                > From: Neeru Walia <neeru.walia@...>
                > Subject: RE: [CMMi Process Improvement] ML4 Glitches
                > To: cmmi_process_improvement@yahoogroups.com
                > Date: Thursday, September 9, 2010, 10:27 AM
                >
                >
                >
                >
                >
                > Hi All,
                >
                >
                >
                > Can somebody please help me on the questions mentioned below?
                >
                >
                > Thanks,
                >
                > Neeru
                >
                >
                > From: cmmi_process_improvement@yahoogroups.com [mailto:cmmi_process_improvement@yahoogroups.com] On Behalf Of Neeru Walia
                > Sent: Wednesday, August 25, 2010 3:22 PM
                > To: cmmi_process_improvement@yahoogroups.com
                > Subject: [CMMi Process Improvement] ML4 Glitches
                >
                >
                >
                > Hi All,
                >
                >
                >
                > This is first time I’m writing to this group and hope I will get expert’s feedbackJ
                >
                >
                >
                > Our organization is going for CMMI ML 4 by next year and by this time we have all the PPMs and PPBs in place. But I have two question here:
                >
                >
                > 1. How often organization should revise the PCB and PPM? Whenever PCBs are revised, is it mandatory to update PPM also?
                >
                >
                >
                > 2. How often Project should run the simulation to validate the success results are within thresholds or not?
                >
                >
                >
                > 3. When a new project has started it will use the Org. PCB and PPM (for similar nature project) to run the simulation and achieve the probability of success? Say if project has its own data points (ex: 15 releases data) then the new project should use their own data(Mean, Stdev) while running simulation or they should add their data point into Org PCB and the new mean and stdev should be used run the simulation?
                >
                >
                >
                > Hope, I have made my question clear.
                >
                >
                > Thanks,
                >
                > Neeru
                >
              Your message has been successfully submitted and would be delivered to recipients shortly.