Loading ...
Sorry, an error occurred while loading the content.

Metrics to Prove XP Works

Expand Messages
  • kcdeberg@excite.com
    We are planning on piloting some XP projects at work, but we continually run across a question that we can t seem to answer effectively: How will you
    Message 1 of 6 , Dec 1, 2000
    • 0 Attachment
      We are planning on piloting some XP projects at work, but we
      continually run across a question that we can't seem to answer
      effectively: "How will you quantitatively show that XP is an
      improvement over more traditional methodologies (iterative
      waterfall)?" We are wondering about this both post-mortem as well as
      in-process. One thought is to use a productivity metric such as
      KAELOC/Staff-month, but as we know, development methodology is only
      one of many factors that work into a productivity metric such as this
      one (others being developer experience, skill level, project
      resources, development environment, etc, etc). Also, since with XP,
      the documentation basically resides in the code with easy-to-read
      refactored code, this could help to inflate those metrics (or
      deflate if prolonged refactoring has occured) depending on the type
      of LOC counter used. After all, I could utilize the CFCO development
      methodology (Code First, Code Often) and produce phenomonal metrics
      which would "compellingly demonstrate" that CFCO is far superior to
      all other methodologies based on KAELOC/Staff-month. Of course, all
      CFCO would do is likely generate a mess of code that is inefficient
      and unmaintainable. Has anyone had to work on establishing some
      measurements that would help to reflect an increase (or decrease) in
      productivity when using an XP methodology? How could this be done so
      that any productivity changes could be reasonably attributed to the
      choice of development methodology, and not to the many other factors
      that effect productivity?

      Bill
    • John Brewer
      ... You might try function points produced, since that might be a closer measure of actual features produced. The trick is that where XP really shines isn t
      Message 2 of 6 , Dec 1, 2000
      • 0 Attachment
        --- In extremeprogramming@egroups.com, kcdeberg@e... wrote:
        > We are planning on piloting some XP projects at work, but we
        > continually run across a question that we can't seem to answer
        > effectively: "How will you quantitatively show that XP is an
        > improvement over more traditional methodologies (iterative
        > waterfall)?"

        You might try function points produced, since that might be a closer
        measure of actual features produced. The trick is that where XP
        really shines isn't how much code is produced, but how much business
        value is delivered. It's very easy (especially in waterfall) to
        produce a project with lots of code, only to have the customer toss
        the whole thing. Big win by "productivity" measures. Big loss by
        "business value added".

        How do you measure productivity improvements when XP isn't involved?

        > KAELOC

        What's this? A KLOC on steroids? A Google search on "KAELOC" didn't
        turn up much.

        John Brewer
        Jera Design
      • Ron Jeffries
        ... If you were considering adding more testing to a team that didn t do much, would you be looking for metrics? If you thought that better manuals would make
        Message 3 of 6 , Dec 1, 2000
        • 0 Attachment
          At 12:13 AM 12/2/00 +0000, kcdeberg@... wrote:
          >We are planning on piloting some XP projects at work, but we
          >continually run across a question that we can't seem to answer
          >effectively: "How will you quantitatively show that XP is an
          >improvement over more traditional methodologies (iterative
          >waterfall)?"

          If you were considering adding more testing to a team that didn't do much,
          would you be looking for metrics? If you thought that better manuals would
          make the customers happier, would you do metrics? Or is the question about
          metrics really someone's question about something else entirely?

          Regards,

          Ronald E Jeffries
          http://www.XProgramming.com
          http://www.objectmentor.com
        • Dossy
          ... If you can afford the time, create one or two pet projects , projects of reasonably small scope, and run two teams to complete the project. One team works
          Message 4 of 6 , Dec 1, 2000
          • 0 Attachment
            On 2000.12.02, kcdeberg@... <kcdeberg@...> wrote:
            > We are planning on piloting some XP projects at work, but we
            > continually run across a question that we can't seem to answer
            > effectively: "How will you quantitatively show that XP is an
            > improvement over more traditional methodologies (iterative
            > waterfall)?" We are wondering about this both post-mortem as well as
            > in-process.

            If you can afford the time, create one or two "pet projects", projects
            of reasonably small scope, and run two teams to complete the project.
            One team works XP, the other in whatever methodology you'd like to
            compare XP to.

            This isn't "quantitative", but as long as the pet projects are
            representative of the kind of projects you expect to take on in
            the future, you'll discover if XP is suitable for your kind of
            projects.

            (The reason why I don't even think quantitative analysis of XP is
            even worthwhile is that there is NO "magic bullet" methodology -
            one methodology may be more suitable for a project than another.
            Even a methodology suitable for a project may not seem quite as
            suitable given other variables: team members' varying skill levels
            and personalities, resources available, geographic location, etc.)


            If you don't have the luxury of time to run this kind of experiment,
            then simply ask: "How did you choose the methodology that [we] are
            currently using?" Then, beg to have your next project done in XP,
            and keep track of only those factors which were used in choosing
            your current methodology, and show using empirical data how XP
            outperforms. ;-)


            - Dossy

            --
            Dossy Shiobara mail: dossy@...
            Panoptic Computer Network web: http://www.panoptic.com/
          • Ron Jeffries
            ... Please tell us why you are interested in piloting XP. That may suggest some metrics. Ronald E Jeffries http://www.XProgramming.com
            Message 5 of 6 , Dec 2, 2000
            • 0 Attachment
              At 12:13 AM 12/2/00 +0000, kcdeberg@... wrote:
              >We are planning on piloting some XP projects at work, but we
              >continually run across a question that we can't seem to answer
              >effectively: "How will you quantitatively show that XP is an
              >improvement over more traditional methodologies (iterative
              >waterfall)?"

              Please tell us why you are interested in piloting XP. That may suggest some
              metrics.

              Ronald E Jeffries
              http://www.XProgramming.com
              http://www.objectmentor.com
            • Kevin Smith
              ... I keep returning to these three: - Customer satisfaction rating - Programmer satisfaction rating - Defects discovered after release Unfortunately, two are
              Message 6 of 6 , Dec 4, 2000
              • 0 Attachment
                On Sat, 02 Dec 2000 00:13:05 +0000 <kcdeberg@...> wrote:

                > We are planning on piloting some XP projects at work, but we
                > continually run across a question that we can't seem to answer
                > effectively: "How will you quantitatively show that XP is an
                > improvement over more traditional methodologies (iterative
                > waterfall)?"

                I keep returning to these three:
                - Customer satisfaction rating
                - Programmer satisfaction rating
                - Defects discovered after release

                Unfortunately, two are "soft". However, they can be
                quantitative if you have a survey: "On a scale of 1 to 10,
                where 1 is miserable and 10 is nirvana, how do you..."

                Perhaps you could return the question to them: "How would you
                quantitatively show that traditional methodologies are more
                effective than XP?"

                > Has anyone had to work on establishing some
                > measurements that would help to reflect an increase (or decrease) in
                > productivity when using an XP methodology? How could this be done so
                > that any productivity changes could be reasonably attributed to the
                > choice of development methodology, and not to the many other factors
                > that effect productivity?

                My boss asked me to come up with metrics that excluded
                "execution" differences. He wanted to isolate the
                differences due to methodology. As folks on this list were
                kind enough to point out, the methodology will directly
                affect the execution, so this isolation is impossible.

                The best you can do is compare similar (or the same,
                sequentially) people working on similar (or the same, in
                parallel) projects using XP and one or more other methods.
                Differences would be presumed to be due to the methodology.

                One other note. If XP has to jump through hoops of fire in
                your organization to be accepted, your organization might
                not be ready for it. The participants and sponsors of an XP
                project have to be open to it, and hope for success. The
                last thing you want on board is someone looking to sink your
                ship, conciously or subconsiously.

                Kevin
              Your message has been successfully submitted and would be delivered to recipients shortly.