Loading ...
Sorry, an error occurred while loading the content.

How to measure agile projects reliability

Expand Messages
  • Md. Abdullah-Al-Mamun
    Hello, Can you please give me some idea about the current practice of Reliability Measurement in agile/eXtreme development. i have listed some possible ways:
    Message 1 of 9 , May 2, 2008
    • 0 Attachment
      Hello,

      Can you please give me some idea about the current practice of
      Reliability Measurement in agile/eXtreme development.

      i have listed some possible ways:

      1. Test Coverage (i.e higher percentage = higher reliability)
      2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
      3. Classical Operational-Profile/User-story

      Eagerly waiting for your valuable replies.

      Thank You.

      /Mamun
      Student, MSc in Soft Eng
      BTH,Sweden
    • D. André Dhondt
      I m not familiar with Goel or Yamada or classical Operational-Profile. I can t speak for Agility or XP in general, but in my experience I ve found there are
      Message 2 of 9 , May 2, 2008
      • 0 Attachment
        I'm not familiar with Goel or Yamada or classical Operational-Profile.

        I can't speak for Agility or XP in general, but in my experience I've found
        there are several lightweight ways to measure reliability--before we deploy
        our applications--with continuous integration, automated acceptance tests,
        and the regular feedback from our customers as we deploy often. After
        deployment, in my understanding of XP, there is no formal measure of
        application reliablity, because we expect the software to work as it was
        designed and tested to work--and we get regular feedback on this expectation
        from the Acceptance Tests, and if they fail, we fix them ASAP. I woudn't
        measure reliability directly unless it was causing problems or it was a
        concern of the customer. On the other hand, I'd never run a production
        process without some server monitoring / alerts configured--but that is so
        easy and inexpensive to configure I don't think it's necessarily the same
        thing you're asking about. What is reliability? Does it include
        high-availability concepts like clustering, disaster recovery, etc?
        Configuring this level of availability would come out of story cards, I
        think.

        On Fri, May 2, 2008 at 7:01 AM, Md. Abdullah-Al-Mamun <mamun222@...>
        wrote:

        > Hello,
        >
        > Can you please give me some idea about the current practice of
        > Reliability Measurement in agile/eXtreme development.
        >
        > i have listed some possible ways:
        >
        > 1. Test Coverage (i.e higher percentage = higher reliability)
        > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
        > 3. Classical Operational-Profile/User-story
        >
        > Eagerly waiting for your valuable replies.
        >
        > Thank You.
        >
        > /Mamun
        > Student, MSc in Soft Eng
        > BTH,Sweden
        >
        >
        >



        --
        D. André Dhondt
        mobile: 267-283-8270
        home: 267-286-6875

        If you're a software developer in the area, join Agile Philly (
        http://groups.yahoo.com/group/agilephilly/)!


        [Non-text portions of this message have been removed]
      • John Roth
        To expand on this a bit: XP doesn t deal with special needs. If you need formal reliability measures you need to add practices to make it happen. That s a
        Message 3 of 9 , May 2, 2008
        • 0 Attachment
          To expand on this a bit: XP doesn't deal with special
          needs. If you need formal reliability measures you need
          to add practices to make it happen. That's a basic
          part of XP: adapting the process to the needs of the
          specific project.

          There are people on this list (and elsewhere) who do
          need formal verification of reliability; many of them
          will be happy to answer your questions of what they
          did.

          John Roth



          ----- Original Message -----
          From: "D. André Dhondt" <d.andre.dhondt@...>
          To: <extremeprogramming@yahoogroups.com>
          Sent: Friday, May 02, 2008 7:45 AM
          Subject: Re: [XP] How to measure agile projects reliability


          I'm not familiar with Goel or Yamada or classical Operational-Profile.

          I can't speak for Agility or XP in general, but in my experience I've found
          there are several lightweight ways to measure reliability--before we deploy
          our applications--with continuous integration, automated acceptance tests,
          and the regular feedback from our customers as we deploy often. After
          deployment, in my understanding of XP, there is no formal measure of
          application reliablity, because we expect the software to work as it was
          designed and tested to work--and we get regular feedback on this expectation
          from the Acceptance Tests, and if they fail, we fix them ASAP. I woudn't
          measure reliability directly unless it was causing problems or it was a
          concern of the customer. On the other hand, I'd never run a production
          process without some server monitoring / alerts configured--but that is so
          easy and inexpensive to configure I don't think it's necessarily the same
          thing you're asking about. What is reliability? Does it include
          high-availability concepts like clustering, disaster recovery, etc?
          Configuring this level of availability would come out of story cards, I
          think.

          On Fri, May 2, 2008 at 7:01 AM, Md. Abdullah-Al-Mamun <mamun222@...>
          wrote:

          > Hello,
          >
          > Can you please give me some idea about the current practice of
          > Reliability Measurement in agile/eXtreme development.
          >
          > i have listed some possible ways:
          >
          > 1. Test Coverage (i.e higher percentage = higher reliability)
          > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
          > 3. Classical Operational-Profile/User-story
          >
          > Eagerly waiting for your valuable replies.
          >
          > Thank You.
          >
          > /Mamun
          > Student, MSc in Soft Eng
          > BTH,Sweden
          >
          >
          >



          --
          D. André Dhondt
          mobile: 267-283-8270
          home: 267-286-6875

          If you're a software developer in the area, join Agile Philly (
          http://groups.yahoo.com/group/agilephilly/)!


          [Non-text portions of this message have been removed]
        • Md. Abdullah-Al-Mamun
          Thank you Mr. John Roth and D. André Dhondt. I have already searched a number of research databases but found a little work on Certification Reliability
          Message 4 of 9 , May 2, 2008
          • 0 Attachment
            Thank you Mr. John Roth and D. André Dhondt. I have already searched a number of research databases but found a little work on Certification Reliability Estimation/Prediction (ofcourse we can say this formal) of agile development.
            I have a little idea what people are doing in the real world regarding this issue and waiting for other peoples here.

            /Mamun


            ----- Original Message ----
            From: John Roth <JohnRoth1@...>
            To: extremeprogramming@yahoogroups.com
            Sent: Friday, May 2, 2008 3:34:23 PM
            Subject: Re: [XP] How to measure agile projects reliability


            To expand on this a bit: XP doesn't deal with special
            needs. If you need formal reliability measures you need
            to add practices to make it happen. That's a basic
            part of XP: adapting the process to the needs of the
            specific project.

            There are people on this list (and elsewhere) who do
            need formal verification of reliability; many of them
            will be happy to answer your questions of what they
            did.

            John Roth

            ----- Original Message -----
            From: "D. André Dhondt" <d.andre.dhondt@ gmail.com>
            To: <extremeprogramming@ yahoogroups. com>
            Sent: Friday, May 02, 2008 7:45 AM
            Subject: Re: [XP] How to measure agile projects reliability

            I'm not familiar with Goel or Yamada or classical Operational- Profile.

            I can't speak for Agility or XP in general, but in my experience I've found
            there are several lightweight ways to measure reliability- -before we deploy
            our applications- -with continuous integration, automated acceptance tests,
            and the regular feedback from our customers as we deploy often. After
            deployment, in my understanding of XP, there is no formal measure of
            application reliablity, because we expect the software to work as it was
            designed and tested to work--and we get regular feedback on this expectation
            from the Acceptance Tests, and if they fail, we fix them ASAP. I woudn't
            measure reliability directly unless it was causing problems or it was a
            concern of the customer. On the other hand, I'd never run a production
            process without some server monitoring / alerts configured-- but that is so
            easy and inexpensive to configure I don't think it's necessarily the same
            thing you're asking about. What is reliability? Does it include
            high-availability concepts like clustering, disaster recovery, etc?
            Configuring this level of availability would come out of story cards, I
            think.

            On Fri, May 2, 2008 at 7:01 AM, Md. Abdullah-Al- Mamun <mamun222@yahoo. com>
            wrote:

            > Hello,
            >
            > Can you please give me some idea about the current practice of
            > Reliability Measurement in agile/eXtreme development.
            >
            > i have listed some possible ways:
            >
            > 1. Test Coverage (i.e higher percentage = higher reliability)
            > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
            > 3. Classical Operational- Profile/User- story
            >
            > Eagerly waiting for your valuable replies.
            >
            > Thank You.
            >
            > /Mamun
            > Student, MSc in Soft Eng
            > BTH,Sweden
            >
            >
            >

            --
            D. André Dhondt
            mobile: 267-283-8270
            home: 267-286-6875

            If you're a software developer in the area, join Agile Philly (
            http://groups. yahoo.com/ group/agilephill y/)!

            [Non-text portions of this message have been removed]




            ____________________________________________________________________________________
            Be a better friend, newshound, and
            know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

            [Non-text portions of this message have been removed]
          • Steven Gordon
            I think we may still be confused about what you are looking for. Are you interested in the reliability of the software being produced or the reliability of the
            Message 5 of 9 , May 2, 2008
            • 0 Attachment
              I think we may still be confused about what you are looking for.

              Are you interested in the reliability of the software being produced
              or the reliability of the project to produce the desired software?

              If you are interested in reliability of the software, are you
              interested in uptime or the chance of the system presenting incorrect
              information or the chance of the system being hacked or what?

              How do non-agile projects do what you are asking about? Do those
              approaches really work in practice or just provide evidence that the
              project team took reasonable measures to address reliability problems?


              On Fri, May 2, 2008 at 7:58 AM, Md. Abdullah-Al-Mamun
              <mamun222@...> wrote:
              >
              >
              > Thank you Mr. John Roth and D. André Dhondt. I have already searched a
              > number of research databases but found a little work on Certification
              > Reliability Estimation/Prediction (ofcourse we can say this formal) of agile
              > development.
              > I have a little idea what people are doing in the real world regarding this
              > issue and waiting for other peoples here.
              >
              > /Mamun
              >
            • Md. Abdullah-Al-Mamun
              I am talking about the developed software. In case of non-agile processes there exists good tools in the market those deal with around a dozen of Software
              Message 6 of 9 , May 2, 2008
              • 0 Attachment
                I am talking about the developed software. In case of non-agile processes there exists good tools in the market those deal with around a dozen of Software Reliability Growth Model (based on the failure data from the testing phase). These model can predict the current reliability level and probable future behavior
                after product release. (Its true, these certification reliability is no practiced by mass organizations but good companies do this)
                In agile the testing phase is not linear like non-agile, there is my problem to apply traditional SRGM. So i like to know what agile/eXtreme ppl are doing?

                /Mamun


                ----- Original Message ----
                From: Steven Gordon <sgordonphd@...>
                To: extremeprogramming@yahoogroups.com
                Sent: Friday, May 2, 2008 6:46:21 PM
                Subject: Re: [XP] How to measure agile projects reliability


                I think we may still be confused about what you are looking for.

                Are you interested in the reliability of the software being produced
                or the reliability of the project to produce the desired software?

                If you are interested in reliability of the software, are you
                interested in uptime or the chance of the system presenting incorrect
                information or the chance of the system being hacked or what?

                How do non-agile projects do what you are asking about? Do those
                approaches really work in practice or just provide evidence that the
                project team took reasonable measures to address reliability problems?

                On Fri, May 2, 2008 at 7:58 AM, Md. Abdullah-Al- Mamun
                <mamun222@yahoo. com> wrote:
                >
                >
                > Thank you Mr. John Roth and D. André Dhondt. I have already searched a
                > number of research databases but found a little work on Certification
                > Reliability Estimation/Predicti on (ofcourse we can say this formal) of agile
                > development.
                > I have a little idea what people are doing in the real world regarding this
                > issue and waiting for other peoples here.
                >
                > /Mamun
                >



                ____________________________________________________________________________________
                Be a better friend, newshound, and
                know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

                [Non-text portions of this message have been removed]
              • Mike Vizdos
                How about working software ? Thank you, Mike On May 2, 2008, at 7:01 AM, Md. Abdullah-Al-Mamun ... [Non-text portions of this message have been removed]
                Message 7 of 9 , May 2, 2008
                • 0 Attachment
                  How about "working software"?

                  Thank you,

                  Mike



                  On May 2, 2008, at 7:01 AM, "Md. Abdullah-Al-Mamun"
                  <mamun222@...> wrote:

                  > Hello,
                  >
                  > Can you please give me some idea about the current practice of
                  > Reliability Measurement in agile/eXtreme development.
                  >
                  > i have listed some possible ways:
                  >
                  > 1. Test Coverage (i.e higher percentage = higher reliability)
                  > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
                  > 3. Classical Operational-Profile/User-story
                  >
                  > Eagerly waiting for your valuable replies.
                  >
                  > Thank You.
                  >
                  > /Mamun
                  > Student, MSc in Soft Eng
                  > BTH,Sweden
                  >
                  >
                  >


                  [Non-text portions of this message have been removed]
                • Steven Gordon
                  On Fri, May 2, 2008 at 11:37 AM, Md. Abdullah-Al-Mamun ... What bad things would happen if you applied the traditional SRGM to the working software delivered
                  Message 8 of 9 , May 2, 2008
                  • 0 Attachment
                    On Fri, May 2, 2008 at 11:37 AM, Md. Abdullah-Al-Mamun
                    <mamun222@...> wrote:
                    >
                    >
                    > I am talking about the developed software. In case of non-agile processes
                    > there exists good tools in the market those deal with around a dozen of
                    > Software Reliability Growth Model (based on the failure data from the
                    > testing phase). These model can predict the current reliability level and
                    > probable future behavior
                    > after product release. (Its true, these certification reliability is no
                    > practiced by mass organizations but good companies do this)
                    > In agile the testing phase is not linear like non-agile, there is my
                    > problem to apply traditional SRGM. So i like to know what agile/eXtreme ppl
                    > are doing?

                    What bad things would happen if you applied the traditional SRGM to
                    the working software delivered after each iteration?

                    I would think you would just get more data points, so I am puzzled by
                    why this would be bad? If the "growth pattern" is different for
                    iterative projects, could you just take this difference into account?

                    Perhaps, it would be too much work to do this every 2 weeks? Could
                    you find ways to automate the majority of the work so that most of the
                    additional work each iteration would just be doing something for only
                    the new pieces of functionality? Could you integrate this into
                    something like FitNesse so that developers, customers, management and
                    other stakeholders can monitor the progress?


                    >
                    > /Mamun
                    >
                    >
                  • Steven Campbell
                    On Fri, May 2, 2008 at 1:37 PM, Md. Abdullah-Al-Mamun ... I ve never heard of SRGMs. It is not a mainstream technique - there isn t even a wikipedia article
                    Message 9 of 9 , May 2, 2008
                    • 0 Attachment
                      On Fri, May 2, 2008 at 1:37 PM, Md. Abdullah-Al-Mamun
                      <mamun222@...> wrote:
                      >
                      > I am talking about the developed software. In case of non-agile processes
                      > there exists good tools in the market those deal with around a dozen of
                      > Software Reliability Growth Model (based on the failure data from the
                      > testing phase). These model can predict the current reliability level and
                      > probable future behavior
                      > after product release. (Its true, these certification reliability is no
                      > practiced by mass organizations but good companies do this)
                      > In agile the testing phase is not linear like non-agile, there is my
                      > problem to apply traditional SRGM. So i like to know what agile/eXtreme ppl
                      > are doing?
                      >

                      I've never heard of SRGMs. It is not a mainstream technique - there
                      isn't even a wikipedia article on it, only a smattering of academic
                      articles. The best summary I could find was
                      http://www.osti.gov/bridge/servlets/purl/6017897-Rc1ams/

                      From what I understand, these models leverage the traditional end of
                      development "QA" step to get baseline measurements, from which a
                      longer-term model can be extrapolated.

                      It would not be applied in an Agile project unless the client attached
                      specific *value* to the SRGM. In that case, we could certainly take
                      measurements at whatever points the models preferred.

                      That said, you would have to be careful before applying a standard
                      model. They were developed using data from projects that "tested
                      quality in". Agile techniques focus on "building the quality in", and
                      so may exhibit different characteristics.

                      --
                      Steve Campbell
                      http://blog.perfectapi.com/
                    Your message has been successfully submitted and would be delivered to recipients shortly.