Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] How to measure agile projects reliability

Expand Messages
  • D. André Dhondt
    I m not familiar with Goel or Yamada or classical Operational-Profile. I can t speak for Agility or XP in general, but in my experience I ve found there are
    Message 1 of 9 , May 2, 2008
    • 0 Attachment
      I'm not familiar with Goel or Yamada or classical Operational-Profile.

      I can't speak for Agility or XP in general, but in my experience I've found
      there are several lightweight ways to measure reliability--before we deploy
      our applications--with continuous integration, automated acceptance tests,
      and the regular feedback from our customers as we deploy often. After
      deployment, in my understanding of XP, there is no formal measure of
      application reliablity, because we expect the software to work as it was
      designed and tested to work--and we get regular feedback on this expectation
      from the Acceptance Tests, and if they fail, we fix them ASAP. I woudn't
      measure reliability directly unless it was causing problems or it was a
      concern of the customer. On the other hand, I'd never run a production
      process without some server monitoring / alerts configured--but that is so
      easy and inexpensive to configure I don't think it's necessarily the same
      thing you're asking about. What is reliability? Does it include
      high-availability concepts like clustering, disaster recovery, etc?
      Configuring this level of availability would come out of story cards, I
      think.

      On Fri, May 2, 2008 at 7:01 AM, Md. Abdullah-Al-Mamun <mamun222@...>
      wrote:

      > Hello,
      >
      > Can you please give me some idea about the current practice of
      > Reliability Measurement in agile/eXtreme development.
      >
      > i have listed some possible ways:
      >
      > 1. Test Coverage (i.e higher percentage = higher reliability)
      > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
      > 3. Classical Operational-Profile/User-story
      >
      > Eagerly waiting for your valuable replies.
      >
      > Thank You.
      >
      > /Mamun
      > Student, MSc in Soft Eng
      > BTH,Sweden
      >
      >
      >



      --
      D. André Dhondt
      mobile: 267-283-8270
      home: 267-286-6875

      If you're a software developer in the area, join Agile Philly (
      http://groups.yahoo.com/group/agilephilly/)!


      [Non-text portions of this message have been removed]
    • John Roth
      To expand on this a bit: XP doesn t deal with special needs. If you need formal reliability measures you need to add practices to make it happen. That s a
      Message 2 of 9 , May 2, 2008
      • 0 Attachment
        To expand on this a bit: XP doesn't deal with special
        needs. If you need formal reliability measures you need
        to add practices to make it happen. That's a basic
        part of XP: adapting the process to the needs of the
        specific project.

        There are people on this list (and elsewhere) who do
        need formal verification of reliability; many of them
        will be happy to answer your questions of what they
        did.

        John Roth



        ----- Original Message -----
        From: "D. André Dhondt" <d.andre.dhondt@...>
        To: <extremeprogramming@yahoogroups.com>
        Sent: Friday, May 02, 2008 7:45 AM
        Subject: Re: [XP] How to measure agile projects reliability


        I'm not familiar with Goel or Yamada or classical Operational-Profile.

        I can't speak for Agility or XP in general, but in my experience I've found
        there are several lightweight ways to measure reliability--before we deploy
        our applications--with continuous integration, automated acceptance tests,
        and the regular feedback from our customers as we deploy often. After
        deployment, in my understanding of XP, there is no formal measure of
        application reliablity, because we expect the software to work as it was
        designed and tested to work--and we get regular feedback on this expectation
        from the Acceptance Tests, and if they fail, we fix them ASAP. I woudn't
        measure reliability directly unless it was causing problems or it was a
        concern of the customer. On the other hand, I'd never run a production
        process without some server monitoring / alerts configured--but that is so
        easy and inexpensive to configure I don't think it's necessarily the same
        thing you're asking about. What is reliability? Does it include
        high-availability concepts like clustering, disaster recovery, etc?
        Configuring this level of availability would come out of story cards, I
        think.

        On Fri, May 2, 2008 at 7:01 AM, Md. Abdullah-Al-Mamun <mamun222@...>
        wrote:

        > Hello,
        >
        > Can you please give me some idea about the current practice of
        > Reliability Measurement in agile/eXtreme development.
        >
        > i have listed some possible ways:
        >
        > 1. Test Coverage (i.e higher percentage = higher reliability)
        > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
        > 3. Classical Operational-Profile/User-story
        >
        > Eagerly waiting for your valuable replies.
        >
        > Thank You.
        >
        > /Mamun
        > Student, MSc in Soft Eng
        > BTH,Sweden
        >
        >
        >



        --
        D. André Dhondt
        mobile: 267-283-8270
        home: 267-286-6875

        If you're a software developer in the area, join Agile Philly (
        http://groups.yahoo.com/group/agilephilly/)!


        [Non-text portions of this message have been removed]
      • Md. Abdullah-Al-Mamun
        Thank you Mr. John Roth and D. André Dhondt. I have already searched a number of research databases but found a little work on Certification Reliability
        Message 3 of 9 , May 2, 2008
        • 0 Attachment
          Thank you Mr. John Roth and D. André Dhondt. I have already searched a number of research databases but found a little work on Certification Reliability Estimation/Prediction (ofcourse we can say this formal) of agile development.
          I have a little idea what people are doing in the real world regarding this issue and waiting for other peoples here.

          /Mamun


          ----- Original Message ----
          From: John Roth <JohnRoth1@...>
          To: extremeprogramming@yahoogroups.com
          Sent: Friday, May 2, 2008 3:34:23 PM
          Subject: Re: [XP] How to measure agile projects reliability


          To expand on this a bit: XP doesn't deal with special
          needs. If you need formal reliability measures you need
          to add practices to make it happen. That's a basic
          part of XP: adapting the process to the needs of the
          specific project.

          There are people on this list (and elsewhere) who do
          need formal verification of reliability; many of them
          will be happy to answer your questions of what they
          did.

          John Roth

          ----- Original Message -----
          From: "D. André Dhondt" <d.andre.dhondt@ gmail.com>
          To: <extremeprogramming@ yahoogroups. com>
          Sent: Friday, May 02, 2008 7:45 AM
          Subject: Re: [XP] How to measure agile projects reliability

          I'm not familiar with Goel or Yamada or classical Operational- Profile.

          I can't speak for Agility or XP in general, but in my experience I've found
          there are several lightweight ways to measure reliability- -before we deploy
          our applications- -with continuous integration, automated acceptance tests,
          and the regular feedback from our customers as we deploy often. After
          deployment, in my understanding of XP, there is no formal measure of
          application reliablity, because we expect the software to work as it was
          designed and tested to work--and we get regular feedback on this expectation
          from the Acceptance Tests, and if they fail, we fix them ASAP. I woudn't
          measure reliability directly unless it was causing problems or it was a
          concern of the customer. On the other hand, I'd never run a production
          process without some server monitoring / alerts configured-- but that is so
          easy and inexpensive to configure I don't think it's necessarily the same
          thing you're asking about. What is reliability? Does it include
          high-availability concepts like clustering, disaster recovery, etc?
          Configuring this level of availability would come out of story cards, I
          think.

          On Fri, May 2, 2008 at 7:01 AM, Md. Abdullah-Al- Mamun <mamun222@yahoo. com>
          wrote:

          > Hello,
          >
          > Can you please give me some idea about the current practice of
          > Reliability Measurement in agile/eXtreme development.
          >
          > i have listed some possible ways:
          >
          > 1. Test Coverage (i.e higher percentage = higher reliability)
          > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
          > 3. Classical Operational- Profile/User- story
          >
          > Eagerly waiting for your valuable replies.
          >
          > Thank You.
          >
          > /Mamun
          > Student, MSc in Soft Eng
          > BTH,Sweden
          >
          >
          >

          --
          D. André Dhondt
          mobile: 267-283-8270
          home: 267-286-6875

          If you're a software developer in the area, join Agile Philly (
          http://groups. yahoo.com/ group/agilephill y/)!

          [Non-text portions of this message have been removed]




          ____________________________________________________________________________________
          Be a better friend, newshound, and
          know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

          [Non-text portions of this message have been removed]
        • Steven Gordon
          I think we may still be confused about what you are looking for. Are you interested in the reliability of the software being produced or the reliability of the
          Message 4 of 9 , May 2, 2008
          • 0 Attachment
            I think we may still be confused about what you are looking for.

            Are you interested in the reliability of the software being produced
            or the reliability of the project to produce the desired software?

            If you are interested in reliability of the software, are you
            interested in uptime or the chance of the system presenting incorrect
            information or the chance of the system being hacked or what?

            How do non-agile projects do what you are asking about? Do those
            approaches really work in practice or just provide evidence that the
            project team took reasonable measures to address reliability problems?


            On Fri, May 2, 2008 at 7:58 AM, Md. Abdullah-Al-Mamun
            <mamun222@...> wrote:
            >
            >
            > Thank you Mr. John Roth and D. André Dhondt. I have already searched a
            > number of research databases but found a little work on Certification
            > Reliability Estimation/Prediction (ofcourse we can say this formal) of agile
            > development.
            > I have a little idea what people are doing in the real world regarding this
            > issue and waiting for other peoples here.
            >
            > /Mamun
            >
          • Md. Abdullah-Al-Mamun
            I am talking about the developed software. In case of non-agile processes there exists good tools in the market those deal with around a dozen of Software
            Message 5 of 9 , May 2, 2008
            • 0 Attachment
              I am talking about the developed software. In case of non-agile processes there exists good tools in the market those deal with around a dozen of Software Reliability Growth Model (based on the failure data from the testing phase). These model can predict the current reliability level and probable future behavior
              after product release. (Its true, these certification reliability is no practiced by mass organizations but good companies do this)
              In agile the testing phase is not linear like non-agile, there is my problem to apply traditional SRGM. So i like to know what agile/eXtreme ppl are doing?

              /Mamun


              ----- Original Message ----
              From: Steven Gordon <sgordonphd@...>
              To: extremeprogramming@yahoogroups.com
              Sent: Friday, May 2, 2008 6:46:21 PM
              Subject: Re: [XP] How to measure agile projects reliability


              I think we may still be confused about what you are looking for.

              Are you interested in the reliability of the software being produced
              or the reliability of the project to produce the desired software?

              If you are interested in reliability of the software, are you
              interested in uptime or the chance of the system presenting incorrect
              information or the chance of the system being hacked or what?

              How do non-agile projects do what you are asking about? Do those
              approaches really work in practice or just provide evidence that the
              project team took reasonable measures to address reliability problems?

              On Fri, May 2, 2008 at 7:58 AM, Md. Abdullah-Al- Mamun
              <mamun222@yahoo. com> wrote:
              >
              >
              > Thank you Mr. John Roth and D. André Dhondt. I have already searched a
              > number of research databases but found a little work on Certification
              > Reliability Estimation/Predicti on (ofcourse we can say this formal) of agile
              > development.
              > I have a little idea what people are doing in the real world regarding this
              > issue and waiting for other peoples here.
              >
              > /Mamun
              >



              ____________________________________________________________________________________
              Be a better friend, newshound, and
              know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

              [Non-text portions of this message have been removed]
            • Mike Vizdos
              How about working software ? Thank you, Mike On May 2, 2008, at 7:01 AM, Md. Abdullah-Al-Mamun ... [Non-text portions of this message have been removed]
              Message 6 of 9 , May 2, 2008
              • 0 Attachment
                How about "working software"?

                Thank you,

                Mike



                On May 2, 2008, at 7:01 AM, "Md. Abdullah-Al-Mamun"
                <mamun222@...> wrote:

                > Hello,
                >
                > Can you please give me some idea about the current practice of
                > Reliability Measurement in agile/eXtreme development.
                >
                > i have listed some possible ways:
                >
                > 1. Test Coverage (i.e higher percentage = higher reliability)
                > 2. Applying Software Reliability Growth Models (like Goel, Yamada etc)
                > 3. Classical Operational-Profile/User-story
                >
                > Eagerly waiting for your valuable replies.
                >
                > Thank You.
                >
                > /Mamun
                > Student, MSc in Soft Eng
                > BTH,Sweden
                >
                >
                >


                [Non-text portions of this message have been removed]
              • Steven Gordon
                On Fri, May 2, 2008 at 11:37 AM, Md. Abdullah-Al-Mamun ... What bad things would happen if you applied the traditional SRGM to the working software delivered
                Message 7 of 9 , May 2, 2008
                • 0 Attachment
                  On Fri, May 2, 2008 at 11:37 AM, Md. Abdullah-Al-Mamun
                  <mamun222@...> wrote:
                  >
                  >
                  > I am talking about the developed software. In case of non-agile processes
                  > there exists good tools in the market those deal with around a dozen of
                  > Software Reliability Growth Model (based on the failure data from the
                  > testing phase). These model can predict the current reliability level and
                  > probable future behavior
                  > after product release. (Its true, these certification reliability is no
                  > practiced by mass organizations but good companies do this)
                  > In agile the testing phase is not linear like non-agile, there is my
                  > problem to apply traditional SRGM. So i like to know what agile/eXtreme ppl
                  > are doing?

                  What bad things would happen if you applied the traditional SRGM to
                  the working software delivered after each iteration?

                  I would think you would just get more data points, so I am puzzled by
                  why this would be bad? If the "growth pattern" is different for
                  iterative projects, could you just take this difference into account?

                  Perhaps, it would be too much work to do this every 2 weeks? Could
                  you find ways to automate the majority of the work so that most of the
                  additional work each iteration would just be doing something for only
                  the new pieces of functionality? Could you integrate this into
                  something like FitNesse so that developers, customers, management and
                  other stakeholders can monitor the progress?


                  >
                  > /Mamun
                  >
                  >
                • Steven Campbell
                  On Fri, May 2, 2008 at 1:37 PM, Md. Abdullah-Al-Mamun ... I ve never heard of SRGMs. It is not a mainstream technique - there isn t even a wikipedia article
                  Message 8 of 9 , May 2, 2008
                  • 0 Attachment
                    On Fri, May 2, 2008 at 1:37 PM, Md. Abdullah-Al-Mamun
                    <mamun222@...> wrote:
                    >
                    > I am talking about the developed software. In case of non-agile processes
                    > there exists good tools in the market those deal with around a dozen of
                    > Software Reliability Growth Model (based on the failure data from the
                    > testing phase). These model can predict the current reliability level and
                    > probable future behavior
                    > after product release. (Its true, these certification reliability is no
                    > practiced by mass organizations but good companies do this)
                    > In agile the testing phase is not linear like non-agile, there is my
                    > problem to apply traditional SRGM. So i like to know what agile/eXtreme ppl
                    > are doing?
                    >

                    I've never heard of SRGMs. It is not a mainstream technique - there
                    isn't even a wikipedia article on it, only a smattering of academic
                    articles. The best summary I could find was
                    http://www.osti.gov/bridge/servlets/purl/6017897-Rc1ams/

                    From what I understand, these models leverage the traditional end of
                    development "QA" step to get baseline measurements, from which a
                    longer-term model can be extrapolated.

                    It would not be applied in an Agile project unless the client attached
                    specific *value* to the SRGM. In that case, we could certainly take
                    measurements at whatever points the models preferred.

                    That said, you would have to be careful before applying a standard
                    model. They were developed using data from projects that "tested
                    quality in". Agile techniques focus on "building the quality in", and
                    so may exhibit different characteristics.

                    --
                    Steve Campbell
                    http://blog.perfectapi.com/
                  Your message has been successfully submitted and would be delivered to recipients shortly.