Loading ...
Sorry, an error occurred while loading the content.

Difference between Smoke & Suspension Testing

Expand Messages
  • vibhanshu bhardwaj
    Dear All, What is the difference between the Smoke Testing and the Suspension Criteria generally we defined before testing as a planning activity. i have
    Message 1 of 61 , Nov 30, 2007
      Dear All,
      What is the difference between the Smoke Testing and the Suspension Criteria generally we defined before testing as a planning activity.
      i have serched the defination on net and found they are similar.
      Is i am correct or not? Please give examples of both type of testing for better understanding.
      Tanks In advance
      Vibhanshu Bhardwaj

      Vibhanshu Bhardwaj

      Get easy, one-click access to your favorites. Make Yahoo! your homepage.
    • Patrick OToole
      Orhan, The fact that Crosby s maturity grid existed before the CMM doe not negate the fact that the then-best practices were identified and then divided into
      Message 61 of 61 , Dec 12, 2007
        The fact that Crosby's maturity grid existed before the CMM doe not negate the fact that the then-best practices were identified and then divided into levels.  True, the CMM didn't "invent" the concept of levels; rather, they borrowed a best practice from other model authors and used that as a framework for their model.  Within this borrowed framework they then placed the practices into the various key process areas, and place the key process areas in the maturity level structure.  (Actually, as Mark's presentation points out, prior to CMM v0.6 they were using a sort of staged/continuous hybrid approach.  The "maturity level" concept jelled with CMM v0.6.)
        I'm not quite sure of your point regarding the fact that practices are not required model components.  Do you perceive that to be a strength or weakness of the model?  As we've discussed before, the practices in the model are those that the model authors thought organizations would perform in order to satisfy the corresponding goal.  However, they didn't want to preclude organizations from satisfying the goals in other ways.  By making the practices "expected," they left open the possibility for alternative practices - alternative ways of performing activities that lead to goal satisfaction.
        The SCAMPI method still relies on direct and indirect artifacts for alternative practice evaluation, as well as affirmations from interviews to make sure that these alternatives practices are serving the projects/organization well.  In other words, an organization cannot say "our alternative practice is to do nothing of this sort"; said another way, the "null set" cannot serve as an alternative practice; or yet another way, practices cannot be made "not applicable."  An organization is EXPECTED to perform the practices in the model OR they will perform some alternative practice that supports the goal at least as well as the model practice or practices not being done.
        As much as I support the idea of alternative practices, the research in ATLAS #10 indicated that they are, by far, the exception not the norm.  This leads one to conclude that the authors did a pretty good job of capturing a broad-based set of "good practices" (I really don't like the term "best practices!") in the model.  They left the door open for alternative practices, but the door is rarely used.
        The model continues to evolve as new "good practices" emerge.  There were about 1500 change requests submitted against CMMI v1.1, and there have been a flurry of them submitted against CMMI v1.2 (I submitted 12 earlier this week and will be submitting another one tomorrow!)  ATLAS #12A proposed 4 significant changes to OPP; ATLAS #12B proposed 4 significant changes to QPM; ATLAS #12C (published yesterday) proposes 7 significant changes to CAR; and ATLAS #12D will propose an as-of-yet-undetermined number of changes to OID.  (There may be an ATLAS #12E which proposes changes to GP4.x and GP5.x as well).  Said plainly, an improvement model must be continuously improved!
        Finally, I wholeheartedly agree with you that the CMMI's structure and content provides an outstanding way for organization's to improve project and organizational performance in a significant and sustainable way.  The evolutionary path provided by the MLs and CLs provides outstanding guidance for a large population of organizations; regardless of their current states, it can help them achieve improved project outcomes today, with the promise of even greater success tomorrow.
        I'm just opposed to saying "the CMMI is the ONLY way" for an organization to improve.
        Have a terrific holiday season!
        ----- Original Message -----
        Sent: Wednesday, December 12, 2007 3:14 PM
        Subject: Re: [CMMi Process Improvement] Re: Process Improvement - What is it?


        I am so happy that we finally found a common ground about my "spirit of CMMI" topic. 

        I have uploaded a file about the history of CMMI by Mark C. Paulk:
        http://tech. groups.yahoo. com/group/ cmmi_process_ improvement/ files/Orhan% 20KALAYCI% 20/Spirit% 20of%20CMMI/

        I am not sure but I have a feeling that after Mark left the CMMI, the spirit begins to diminish.  I do not have any evidence for that it is just a feeling. 

        See page 3, slide 6 "Inspirations for the Software CMM"
        - Dissatisfaction with known, consistent software problems
        - Total Quality Management (TQM) successes
        - Crosby's maturity grid
        - IBM's process grid

        The famous 5 levels of maturity can be seen in Crosby's maturity grid (1979) and IBM's process grid (1985).  There were there before the CMM.  So it is not true that the best practices were identified and then for the ease of management they had been divided into groups called levels.  :) 

        What do you say about my claim about "practices are not even required components of the model.  Isn't it an overestimating the value of practices saying "CMMI is a best practice model"? 

        This however explains why you and most experts say CMMI is just one of the many models that guide organizations for process improvement.  If you consider CMMI is a list of best practices of course there are many more models can be found listing other best practices. 

        CMMI, for me, is unique in defining the natural evolutionary PATH (levels) for process improvement.  Especially CL1 and CL2 is very interesting that I have not seen any other model suggesting to start by individual and group (project) maturity first. 

        As you said, for a CL1 capability PSP and CL2 capability TSP can be very much useful. 

        My definition for CMMI is "CMMI defines a PATH to follow for quantitative and continues process improvement.  The path is defined as levels to climb.  Skipping levels can be counterproductive.  Especially nowadays, too many organizations try to jump to CL3 or ML3 without experiencing CL2 or ML2.  They do not know the importance of CL2 or CL1 and nobody explains to them. 

        One can not explain the importance of CL1 and CL2 without the spirit (meaning of the levels). 

        May the peace be upon you,

        2007/12/8, Patrick OToole <PACT.otoole@ att.net>:

        I don't know if you are aware of it or not, but you're (somewhat tongue-in-cheek) description of how the CMMI's list of best practices was just divided into 5 different levels is pretty close to the way Bill Curtis described it to me!
        When the SEI decided to evolve form a maturity questionnaire to a model, they invited a group of what were perceived as higher maturity organizations to Pittsburgh, and told them to be prepared to discuss the practices that make them as good as they are.  They sat in a conference room and wrote their good practices on Post-It Notes (or equivalent) and plastered the walls with them.
        They then went through an affinity diagramming exercise - grouping the practices into clumps of like activities - project management, engineering, etc.  They then went through a similar exercise for each clump, ultimately winding up with groups that served as the foundations for the CMM's key process areas which were then place into "maturity levels" based on what the folks felt should be addressed first.
        So you were right - they just kinda broke the good practices into levels.
        I DO think it's fair to say that the CMMI is a collection of "best practices."  After all, the model authors wouldn't purposely design a model with "suboptimal" or "good enough" practices.  Rather, as was done originally, they are looking to identify those practices that heighten the probability of an organization achieving success in the marketplace.  Not a bad objective!
        I'd like to share one other bit of historical trivia that I "ran into" recently.  A couple of months back, Watts and I were in Pittsburgh together and went out for an early morning run.  In discussing the CMMI, he mentioned that ML2's focus on project management wasn't really his original vision of what ML2 should be!  Rather, he envisioned the practices of SPP & SPTO (Software Project Planning &  Software Project Tracking and Oversight) to be preformed by the ENGINEERS, not by a project manager!  Get the people that are going to perform the work to generate the estimates, identify the risks, assign the tasks and responsibilities, and commit to the plan.  This REALLY makes sense to me in view of his most recent work on TSP!
        So all this time I, and everyone else that I know in the business of CMMI-based process improvement, have been professing the idea that ML2 is all about project management, when it was REALLY intended (at least in Watts' mind) to be about the people who were going to do the work, planning and tracking their work ala TSP light!  Who would have thought!
        Anyway, whether the "spirit" is embedded in the model itself as you contend, or is injected by the people using the model to improve the way they perform the work as I contend, is less relevant than agreeing that the model is best used when helping an organization IMPROVE, rather than merely helping an organization meet a contractual obligation by generating a bunch of artifacts to show that they have obtained the requisite level.
        I hope that we have, at long last, found common ground!
        ----- Original Message -----
        Sent: Friday, December 07, 2007 7:26 PM
        Subject: Re: [CMMi Process Improvement] Re: Process Improvement - What is it?


        Thank you for supporting my positions!  :)  [I think we both have an amazing ability to understand other party as we wish :) ]

        I thank you because you said:  CMMI is a list of good practices.  In fact, today I was reshuffling my folders and came across your presentation in NDIA 2007:  "Judging the Suitability of Alternative Practices"  (by you and Rick Hefner from Northrop Grumman) 

        You are consistent (stable :)  for your definition for CMMI.  Your definition in the paper is the same:  CMMI is a best practice model.  I see you approached the alternative practices as although CMMI already list the best practices they may not be applicable for all contexts.  So, there may be some contexts need alternative practices.  It is very logical if we assume "CMMI is a best practice model"  or "CMMI is the list of best practices" 

        Levels then probably will be explained as just some sub sets of best practices.  Since the number of best practices are so big that we need to prioritize them as dividing them into four groups called levels :))  That is why there are levels in the CMMI isn't it? 

        It is amazing how CMMI has been reduced to a list of best practices where practices are even not required components of the model.  (Pat is not the only one who defines the CMMI in this way, even some official presentations in SEI site say the same thing :) 

        As I always said, CMMI is not only a list of (best) practices.  It has a spirit. Although it is hidden and not communicated well enough, today. Still there is one.  Otherwise it cannot survive :)   The spirit is the levels.  Levels exists  as stand alone entities they do not need practices to be understood.  I can explain them without using any practice or even without help of any goal.  As I recently (yesterday) did :)

        Thanks to you, Pat, it is now completely obvious that CMMI has been reduced to just some list of (best) practices for sometime.  That is why it is not that clear why only a process at CL4 can be subject to quantitative process improvement, i.e., only after CL3, 6-sigma and the most models are applicable.  There are some models begin to exist sharing the same spirit (levels) with CMMI such as PSP, TSP, Cobit, OPM3 etc. 

        Thanks should also go to you Pat, because you also explained how SCAMPI is away from the spirit of CMMI.  You were saying, although a process is stable we can not give it the rating CL4.  It is against the spirit.  It is either not stable or really not deserve the CL4.  You should not feel comfortable by saying  although it is stable we cannot say it at CL4 because of SCAMPI A requirements.  This is another clear sign of missing spirit. 

        And that is why I suggest to separate CMMI into at least two parts:  (1) Levels (Spirit) and (2) implementation guidelines (best practices) for different context such as DEV, ACQ, SVC, and sub domains within these contexts such as agile development, etc. 

        May the peace be upon you,
        http://www.xpi. ca/OrhanKALAYCI. ppt
        http://www.linkedin .com/in/orhankal ayci
        http://www.sei. cmu.edu/iprc/ ipssmembers. html

        2007/12/7, Patrick OToole < PACT.otoole@ att.net>:

        I contend that a model has no "spirit" in isolation.  It's just a list of good practices that many organizations find useful.  The "spirit" is injected (or "deflated") by the people who use the model.  If you use the model as the basis for improving the way the work is performed, you have used it in the "spirit" of the author's intentions.  If you simply use it to get a level, the "spirit" is limited to the alcoholic beverages that you consume as you celebrate your hollow victory.
        I have an upcoming ATLAS publication that will make a rather revolutionary suggestion related to high maturity practices - I think you, in particular, will enjoy the "spirit" of the proposed changes!  (Note: This is not ATLAS #12C - proposed changes to CAR - which I just pre-published to my review committee today.  If anyone is interested in participating in the ATLAS #12C study, send me an email at PACT.otoole@ att.net).
        Regarding statistical stability, I am agreeing with you that it is a concept tightly aligned with CL4.  However, it is possible to have statistical stability at lower APPRAISED capability levels due to the stringent requirements of the SCAMPI appraisal method.  If all practices are performed and institutionalized EXCEPT for having an organizational policy, the process area would be APPRAISED at CL1 - DESPITE the fact that the process is being performed "in the spirit" of CL4!  Telcordia had a number of CMM Key Process Areas that were of this ilk - the practices were being performed but there were a few minor holes in the generic practices that would have resulted in a CL1 rating. (But, hey, that's why we do Class C and Class B appraisals, right?)
        I have no problem with your analysis of using the CMMI to achieve stable process and ultimately, optimizing processes.  I just take exception to your statement that it's the ONLY way to reach this point.  Especially since I have experience with organizations that used Six Sigma, PSP/TSP, ISO9000, and Malcolm Baldridge to accomplish this same feat.  I make my living from providing CMMI-related services, but I would lose credibility if I told people that I have the SOLE answer to their problems.  Hell, I would lose credibility with MYSELF!
        ----- Original Message -----
        Sent: Thursday, December 06, 2007 11:37 PM
        Subject: Re: [CMMi Process Improvement] Re: Process Improvement - What is it?



        Let's, you and me, may stop talking about the topic but do not forget there are more than four thousands members in the list.  There may be 1 out of thousand (i.e. four) people want to talk about it :)


        I see, you do not want to talk about my claim that "CMMI is the only PATH for process improvement" any more.  OK. 


        I would like to continue to talk to you about "stability" and quantitative process improvement. 


        In fact, I will call it "missing spirit of CMMI" again :)  I mean the purpose or meaning of the levels are still not clear.  You are saying there is stability before the CL4, that is to say, we can measure the process and count on it at lower CLs.  You are defining here another kind of stability i.e. a consistent way doing business; you noted it as "remember ML2 was named as repeatable"


        Let me make it clear:  There is no other definition of stability.  Stability is stability.  :)  Consistent way of doing a work is the (plain English) definition of it.  Having no special causes of variation is the quantitative (mathematical) definition of it.  Stability is doing the things in a consistent way and then measuring the variation to make sure they are performed in a consistent way.  This is to say, we are at the level of certainty (i.e. level five) in Crosby's maturity grid. 


        You are right, in some sense; there is consistency in lower levels of CL.  However, the scope of the consistency in lower levels is very limited.  Namely, in CL1 the consistency is limited to just one person :) everybody is capable of producing output in his or her way of doing the things.  The process at CL1 is consistent for each and every individual in a different way.  That is true; this is a kind of consistency. The process at CL2 is consistent within the project.  There may be different processes for each project.  Nobody can guarantee that the process does continue to exist when the project is over.  This kind of consistency make the process is at CL2.  At CL3, there is a real consistent way of doing business within the whole organization.  Everybody in the organization do the same thing in the same way though there may be few different main streams to choose from, such as small, medium, big projects and/or domain1, domain2, domain3 projects.  OK, at CL3 there is a huge consistency right now we are ready to measure the process and make sure it is really stable.  Making sure process at CL3 is really stable will make it at CL4.  And we can continue with continues quantitative process improvement at CL5. 


        In fact, it is easier to read in reverse order:

        We want to improve our process quantitatively and continuously (CL5)

        To do so, we need the process to be stable (CL4)

        To do so, we need the process consistently performed within the whole Organization (CL3)

        To do so, let's divide and conquer the problem,

        First, we target individual consistency (one by one, people performing the process) (CL1)

        Latter, we target the consistency within the projects (project by project) (CL2)


        THAT IS THE SPIRIT of CMMI and clearly it is MISSING. 


        May the peace be upon you,


        2007/12/6, Patrick OToole <PACT.otoole@ att.net>:

        I think at this point it's probably best to "agree to disagree" and move on to other topics ripe for vigorous debate!
        BTW, I hope to see you at the SEPG Conference in March (in Tampa).
        ----- Original Message -----
        Sent: Thursday, December 06, 2007 8:12 AM
        Subject: Re: [CMMi Process Improvement] Re: Process Improvement - What is it?


        Thank you for strongly supporting your positions!  :))

        I believe there was gravity before Newton found it!  :) 

        After Newton found an explanation why objects fall towards to ground, it was the only theory about the forces of nature since there was no observation or no need to know about nuclear power, electromagnetic power, radiation power.  So, for a long time, Newtonian physic was the only theory about the nature and it was perfectly working. 

        Today, I believe the CMMI is the only theory about natural evolution of a process towards its maturity.  I do not know any other theory exists for explaining the evolutionary path for maturity of capability of a process. 

        Only after CL4, our process is ready for mathematical analysis and improvement before that there is only CLs of CMMI who guides us towards maturity of capability of a process.  Business results can be improved by several ways but unless it is a process improvement at the same time these improvements will not last long.  Process improvement can only be measured by CLs of CMMI before the CL4.  It is not to say we do need to measure business results before CL4 but they are not improvements for process improvement.  That is why there is MA in ML2. 

        May the peace be upon you,
        http://www.linkedin .com/in/orhankal ayci
        http://www.sei. cmu.edu/iprc/ ipssmembers. html

        2007/12/5, Patrick OToole < PACT.otoole@ att.net>:

        You have the amazing ability to select only those portions of a post that match your opinions and ignore everything else!
        It was pretty obvious that Jeffry, self-admittedly new to CMMI, was not using "stable process" in the statistical sense.  Rather, he was using it in a way that is more reflective of ML2 and ML3 - performing the process in a reasonably consistent way (remember that we used to call ML2, "Repeatable").  You also ignored the end of his post where he explicitly stated his disagreement with your initial premise, that processes performing at less than ML4 cannot be stable.  And yet you use his disagreeing post as support for your position!
        When Dr. Bill Curtis (PhD in Statistics) conducted the initial CMM ML5 appraisal at Telcordia, he concluded that MOST of the practices were implemented in a way that was consistent with the model through level 5.  Two things of note: first - Telcordia had barely heard of the CMM before Bill's visit.  They used their ISO9000 program to instill process discipline.  It was their ISO auditor who suggested that they may want to take a look at the CMM since he perceived that their practices aligned very nicely with that model.
        Second, the elements that prevented them from being appraised at ML5 were mostly infrastructure in nature - they didn't have policies in place (despite their President's STRONG advocacy for process discipline), and they were missing some of the CMM-recommended "documented procedures."  Other than that, they were there!
        So Telcordia got to ML5 without using the CMM as the basis for its improvement program.  And, without policies in place, the highest they would have been appraised against the CMMI in any given process area would have been CL1.  So you do NOT have to have CL4 processes to achieve stability, or process capability, or quantitative management.  They had the stuff in place long before they heard of the CMMI.  (Bill, Norm Hammock, and I co-led the CMM appraisal when Telcordia ultimately achieved their ML5 rating).
        I also led the ML5 CMM appraisal for the IBM facility in Rochester, Minnesota.  They had won the national Malcolm Baldridge Award circa 1991 - again prior to having ever HEARD of the CMM.  When I conducted their ML4 Class B-line CMM appraisal back in 2004, they were relatively new to the CMMI, and they certainly hadn't done anything to the quantitatively managed processes that they had in place long before they first picked the model up.  (Full Disclosure - Steve Kan in the Quality Manager at IBM Rochester, and also the author of the book, "Metrics and Models in Software Quality Engineering."  The SEI uses one of Steve's charts from the book in Module 10 of the "Intro to CMMI" course.  Fuller Disclosure - Capers Jones and I each guest wrote a chapter in the second edition of Steve's book).
        In most high maturity appraisals, ML2 and ML3 and fairly strong, and ML4 and ML5 (but ESPECIALLY ML4) are relatively weak.  At IBM Rochester, the situation was exactly the opposite.  ML4 was glorious!  Their charts were exceptional and the data DROVE their development process.  On the other hand, ML2 and ML3 had some issues that needed to be addressed before we conducted the formal CBA IPI six months later, but ML4 was clearly there.  Initially, they didn't want to put ML5 in scope until we conducted a "micro-assessment" and determined that it, too, was very low risk for the formal appraisal.
        So Telcordia used ISO9000, IBM used the Malcolm Baldridge Award, and my most recent ML5 appraised company (yet undisclosed) used Six Sigma in conjunction with CMMI.  From my relatively limited experience with high maturity organizations, I simply cannot accept either of your premises: that CLs are the only path to high maturity and that until a process is statistically stable (CL4 or otherwise), measureable improvement cannot be made.
        Somehow I fear that you will respond to this post by thanking me for strongly supporting your positions!
        ----- Original Message -----
        Sent: Wednesday, December 05, 2007 8:08 AM
        Subject: Re: [CMMi Process Improvement] Re: Process Improvement - What is it?

        Thanks God :)

        At last, someone is talking the same language... 

        Thank you Jeffry.  Without a stable process we can not talk about "quantitative process improvement" 

        CMMI CL4 is where our process is stable.  It is the definition of CL4.  If a process is stable it means that the process is at CL4.  Consequently, at CL3, 2, 1 the process is unstable and is not ready for 6-sigma or any other quantitative improvement.  At CL3,2,1 a process can not be stable.  Please see that if a process is stable it means that it is at CL4 otherwise it is not at CL4 but a lower CL.  That is why a process at CL3,2,1 can be improved using 6-sigma.  You may know,  I was saying "the spirit is missing" meaning that the real definition or characteristics of levels are not emphasized enough today, they are not known.  So, how can we assume the spirit is communicated well, just the opposite, the spirit is getting diminish each and every day because even experts are saying against the spirit ( i.e. meaning & purpose of CMMI).  CMMI defines the characteristics of mature processes, the PATH. It is the only PATH that I know for process improvement.  However, it does not define HOW to reach a higher level of maturity [walk on the PATH].  After CL3, 6-sigma defines very good HOW to reach higher CLs. 

        So after CL4 any improvement model is applicable but before than that only CLs of CMMI will help for process improvement.  (Business objectives are always important for process improvement but without CLs, the results will be just seasonal (not long lasting) improvements only - See the seven secrets of process improvement of mine for the importance of business objectives such as less cost, less duration, less defects, more functionality - http://www.xpi. ca/NDIA2007/ SevenSecretsBind le.pdf )

        Sorry about my offensiveness, I am already under attack :) 

        I do respect the experts, I would like them to explain the more PATHs for process improvement other than CLs.  Please do not just say: There are others.  Please name them and explain how they help organization to reach stable processes so that they can continue improvement with quantitative methods such as 6-sigma. 

        May the PEACE be upon you,
        http://www.linkedin .com/in/orhankal ayci
        http://www.sei. cmu.edu/iprc/ ipssmembers. html

        2007/12/5, Jeffry Yoris < jyoris@balicamp. com>:

        I'm newbie with CMMI model. But its obvious that you cannot start
        improvement on unstable process. In Six Sigma DMAIC approach, one of the
        goals of Measurement stage (the "M" part) is to measure how stable is
        the current process [using: Cp, Cpk, control chart, etc]. If its not
        stable, do not proceed until it is.

        But I don't agree that below level 4, ALL processes within an
        organization are not stable - therefore you cannot do any improvement.
        Pls let me know what you think.


        ------------ --------- --------- --------- --------- ---------
        Re: Process Improvement - What is it?
        Posted by: "Orhan KALAYCI" orhan.kalayci@ xpi.ca orhankalayci2001
        Tue Dec 4, 2007 3:21 pm (PST)
        Thank you Bruce,

        I am fine. I am now in Toronto and looking for a full time position. If
        you know any opportunity in Canada please let me know via private email.

        I see your point regarding to MA. I believe MA is very useful and value
        added activity however I do not believe it can be used to measure
        in process improvement for processes with a CL lower than 4. It is just
        because if a process is not stable, which is the case before CL4, any
        improvement in a project most probably will not pass to next project. I
        would love to hear you taking about "importance of stability and its
        relevance with process improvement" "how stability can be achieved"

        May the peace be upon you,
        http://www.linkedin .com/in/orhankal ayci
        http://www.sei. cmu.edu/iprc/ ipssmembers. html

      Your message has been successfully submitted and would be delivered to recipients shortly.