Loading ...
Sorry, an error occurred while loading the content.

Choice modeling and Agile?

Expand Messages
  • Jeff Lash
    I wanted to see if anyone was using choice modeling in conjunction with Agile development? For those not familiar with it, choice modeling views the value of a
    Message 1 of 12 , Jun 30, 2005
    • 0 Attachment
      I wanted to see if anyone was using choice modeling in conjunction
      with Agile development?

      For those not familiar with it, choice modeling views the value of a
      product (and the price one is willing to pay for it) as the sum of the
      features and attributes the product contains. You ask participants not
      just to rank elements in priority order, but instead to assign dollar
      values to features based on their importance to the customer. The more
      dollars assigned to a feature/attribute, the more important it is to
      the customer.

      The resulting data can be used not only to understand what is
      preferred, but how valuable each element is, how much more important
      certain elements are than others, and potentially how much a customer
      may be willing to pay for a product with a given set of features and
      attributes.

      It's actually much more than that -- there's a good body of
      statistical, econometric, and business academia work in the field --
      but that's the simple, practical application of the idea.

      This seems to be a natural fit with the prioritization of features
      that goes on for each iteration. I'm trying it out, combining the
      results of choice modeling and traditional questionnaires (which
      capture what people say they want) with data gathered during field
      studies and ethnographic interviews (which captures what people really
      need, often unspoken), and using all of that to better understand
      prioritization. It's certainly a "lightweight" approach to choice
      modeling, but just trying it out on a project, it seems to have
      yielded some interesting insights.

      So, I wanted to see if anyone has tried anything similar, or has any
      thoughts on this approach.

      Jeff

      jeff@...
    • acockburn@aol.com
      I ve tried, but can t find anyone who can come up with the numbers. How do you get those detailed number? Alistair In a message dated 7/2/2005 9:02:02 A.M.
      Message 2 of 12 , Jul 2, 2005
      • 0 Attachment
        I've tried, but can't find anyone who can come up with the numbers.
        How do you get those detailed number?
        Alistair
         
        In a message dated 7/2/2005 9:02:02 A.M. Mountain Daylight Time, agile-usability@yahoogroups.com writes:
        Subject: Choice modeling and Agile?

        I wanted to see if anyone was using choice modeling in conjunction
        with Agile development?

        For those not familiar with it, choice modeling views the value of a
        product (and the price one is willing to pay for it) as the sum of the
        features and attributes the product contains. You ask participants not
        just to rank elements in priority order, but instead to assign dollar
        values to features based on their importance to the customer. The more
        dollars assigned to a feature/attribute, the more important it is to
        the customer.

        The resulting data can be used not only to understand what is
        preferred, but how valuable each element is, how much more important
        certain elements are than others, and potentially how much a customer
        may be willing to pay for a product with a given set of features and
        attributes.

        It's actually much more than that -- there's a good body of
        statistical, econometric, and business academia work in the field --
        but that's the simple, practical application of the idea.

        This seems to be a natural fit with the prioritization of features
        that goes on for each iteration. I'm trying it out, combining the
        results of choice modeling and traditional questionnaires (which
        capture what people say they want) with data gathered during field
        studies and ethnographic interviews (which captures what people really
        need, often unspoken), and using all of that to better understand
        prioritization. It's certainly a "lightweight" approach to choice
        modeling, but just trying it out on a project, it seems to have
        yielded some interesting insights.

        So, I wanted to see if anyone has tried anything similar, or has any
        thoughts on this approach.

        Jeff
         
        ==============================================
        Alistair Cockburn
        President, Humans and Technology

        801.582.3162
        1814 Ft Douglas Cir,
        Salt Lake City, UT 84103
        http://alistair.cockburn.us/
        acockburn@...
        ===============================

        "Surviving Object-Oriented Projects" (1998)
        "Writing Effective Use Cases" (Jolt Productivity Award 2001)
        "Agile Software Development" (Jolt Productivity Award 2002)
        "Crystal Clear: A Human-Powered Methodology for Small Teams" (Jolt Award Finalist 2004)

        "La perfection est atteinte non quand il ne reste rien a ajouter,
        mais quand il ne reste rien a enlever." (Saint-Exupery)

        "The first thing to build is trust." (Brad Appleton)
        ==============================================

      • Adi The
        Jeff, Thanks for sharing the information about the choice modeling. One of our researchers conducted a similar workshop session called Idea Shopping after
        Message 3 of 12 , Jul 2, 2005
        • 0 Attachment
          Jeff,
          Thanks for sharing the information about the choice modeling.
          One of our researchers conducted a similar workshop session called
          "Idea Shopping" after
          preceding ethnographic field work results analysed and synthesised.
          For the "Idea Shopping" workshop, several UI paper prototypes, such as
          login UI prototypes, were prepared.
          Each prototype was assigned some "Kroner" values (Scandinavian
          currency) based on the analysis and synthesis of field work results
          and
          each workshop participant was given some "pocket money" to shop for
          the prototypes.
          The workshop participants could shop until they spent all their "money".
          During the shopping, the participants could provide instant feedback,
          ask questions, have dialogues with the other shoppers, etc.
          The results of the workshop were then used to narrow down the choices
          and refine the prototypes.

          --
          Adi B. Tedjasaputra

          IT and User Experience Manager
          TRANSLATE-EASY
          :: RFID and Human-centred Design Innovation Centre in Asia ::
          http://TRANSLATE-EASY.com

          Menara Kadin Indonesia 30th Floor
          Jl. Rasuna Said Block X-5, Kav 2-3
          Jakarta 12950
          INDONESIA

          Direct. +45 - 20 27 77 53
          Fax. +45 - 28 17 09 78
        • Lynn Miller
          In the poast we have used choice modeling during the product validation stage when we haven t got down to the details yet. Once down to the level of features,
          Message 4 of 12 , Jul 4, 2005
          • 0 Attachment
            In the poast we have used choice modeling during the product validation
            stage when we haven't got down to the details yet.

            Once down to the level of features, it often makes sense to use a
            similar but reversed method - you assign dollar values to the features
            and give the users a certain amount to spend.

            The nice thing about this second method is that you can balance the
            development costs of features by price. That is, a huge feature isn't
            given the same weight as a small feature because it costs more. So
            users can pick the big ones that are really important and mix in some of
            the smaller ones. The amount they have to spend depends on your release
            date so this works better with fixed release dates.

            I'm glad to see that you're combining the results with other data
            gathering methods. On its own this method doesn't produce full
            workflows - it just helps you to focus on what is important. And you
            have to have done the work to get the list down to a reasonable size
            before it goes in front of users.

            Lynn Miller

            Jeff Lash wrote:
            > I wanted to see if anyone was using choice modeling in conjunction
            > with Agile development?
            >
            > For those not familiar with it, choice modeling views the value of a
            > product (and the price one is willing to pay for it) as the sum of the
            > features and attributes the product contains. You ask participants not
            > just to rank elements in priority order, but instead to assign dollar
            > values to features based on their importance to the customer. The more
            > dollars assigned to a feature/attribute, the more important it is to
            > the customer.
            >
            > The resulting data can be used not only to understand what is
            > preferred, but how valuable each element is, how much more important
            > certain elements are than others, and potentially how much a customer
            > may be willing to pay for a product with a given set of features and
            > attributes.
            >
            > It's actually much more than that -- there's a good body of
            > statistical, econometric, and business academia work in the field --
            > but that's the simple, practical application of the idea.
            >
            > This seems to be a natural fit with the prioritization of features
            > that goes on for each iteration. I'm trying it out, combining the
            > results of choice modeling and traditional questionnaires (which
            > capture what people say they want) with data gathered during field
            > studies and ethnographic interviews (which captures what people really
            > need, often unspoken), and using all of that to better understand
            > prioritization. It's certainly a "lightweight" approach to choice
            > modeling, but just trying it out on a project, it seems to have
            > yielded some interesting insights.
            >
            > So, I wanted to see if anyone has tried anything similar, or has any
            > thoughts on this approach.
            >
            > Jeff
            >
            > jeff@...
          • Robin Dymond
            That valuation dilemma is exactly the problem I have on one current project. The client has a long list of features for an Intranet, their priorities, and a
            Message 5 of 12 , Jul 4, 2005
            • 0 Attachment
              That valuation dilemma is exactly the problem I have on one current project. The client has a long list of features for an Intranet, their priorities, and a limited budget. They know the features are valuable to the users, but they don't know how valuable, therefore they don't know how to come up with an ROI. Even if they did come up with an ROI (a valuable exercise for dealing with CFOs) the assumptions in the ROI are WAGs (wild ass guesses). As with most projects, the wish list far exceeds the funding available.
               
              Another project has a very clear ROI: reduce call center calls by X %, and reduce costs per interaction by Y%. These are great metrics to prioritize and build against, because the guidance is clear for everyone. It also gives guidance on budget available and feasibility.
               
              cheers,
              Robin Dymond

              From: agile-usability@yahoogroups.com [mailto:agile-usability@yahoogroups.com] On Behalf Of acockburn@...
              Sent: Saturday, July 02, 2005 10:15 AM
              To: agile-usability@yahoogroups.com
              Subject: [agile-usability] Re: Choice modeling and Agile?

              I've tried, but can't find anyone who can come up with the numbers.
              How do you get those detailed number?
              Alistair
               
              In a message dated 7/2/2005 9:02:02 A.M. Mountain Daylight Time, agile-usability@yahoogroups.com writes:
              Subject: Choice modeling and Agile?

              I wanted to see if anyone was using choice modeling in conjunction
              with Agile development?

              For those not familiar with it, choice modeling views the value of a
              product (and the price one is willing to pay for it) as the sum of the
              features and attributes the product contains. You ask participants not
              just to rank elements in priority order, but instead to assign dollar
              values to features based on their importance to the customer. The more
              dollars assigned to a feature/attribute, the more important it is to
              the customer.

              The resulting data can be used not only to understand what is
              preferred, but how valuable each element is, how much more important
              certain elements are than others, and potentially how much a customer
              may be willing to pay for a product with a given set of features and
              attributes.

              It's actually much more than that -- there's a good body of
              statistical, econometric, and business academia work in the field --
              but that's the simple, practical application of the idea.

              This seems to be a natural fit with the prioritization of features
              that goes on for each iteration. I'm trying it out, combining the
              results of choice modeling and traditional questionnaires (which
              capture what people say they want) with data gathered during field
              studies and ethnographic interviews (which captures what people really
              need, often unspoken), and using all of that to better understand
              prioritization. It's certainly a "lightweight" approach to choice
              modeling, but just trying it out on a project, it seems to have
              yielded some interesting insights.

              So, I wanted to see if anyone has tried anything similar, or has any
              thoughts on this approach.

              Jeff
               
              ==============================================
              Alistair Cockburn
              President, Humans and Technology

              801.582.3162
              1814 Ft Douglas Cir,
              Salt Lake City, UT 84103
              http://alistair.cockburn.us/
              acockburn@...
              ===============================

              "Surviving Object-Oriented Projects" (1998)
              "Writing Effective Use Cases" (Jolt Productivity Award 2001)
              "Agile Software Development" (Jolt Productivity Award 2002)
              "Crystal Clear: A Human-Powered Methodology for Small Teams" (Jolt Award Finalist 2004)

              "La perfection est atteinte non quand il ne reste rien a ajouter,
              mais quand il ne reste rien a enlever." (Saint-Exupery)

              "The first thing to build is trust." (Brad Appleton)
              ==============================================


              The information contained in this message is confidential. It is intended to be read only by the individual or entity named above or their designee. If the reader of this message is not the intended recipient, you are hereby notified that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete or destroy any copy of this message.
            • Ron Jeffries
              ... Well, of course they should be able to express the ROI. But I d suggest that it s not necessary . Given two features, if they can decide which one to do
              Message 6 of 12 , Jul 4, 2005
              • 0 Attachment
                On Monday, July 4, 2005, at 4:16:15 PM, Robin Dymond wrote:

                > That valuation dilemma is exactly the problem I have on one
                > current project. The client has a long list of features for an
                > Intranet, their priorities, and a limited budget. They know the
                > features are valuable to the users, but they don't know how
                > valuable, therefore they don't know how to come up with an ROI.
                > Even if they did come up with an ROI (a valuable exercise for
                > dealing with CFOs) the assumptions in the ROI are WAGs (wild ass
                > guesses). As with most projects, the wish list far exceeds the
                > funding available.

                Well, of course they "should" be able to express the ROI. But I'd
                suggest that it's not "necessary".

                Given two features, if they can decide which one to do first, that's
                often "enough".

                Sometimes folks have trouble doing that. I help them along by
                picking something obviously valuable and something obviously dull.
                (There are always features that qualify.) Then I suggest that we do
                the dull one first, and defer the valuable one for a long time. They
                call me an idiot, and get about the business of deciding what to do
                next.

                > Another project has a very clear ROI: reduce call center calls by
                > X %, and reduce costs per interaction by Y%. These are great
                > metrics to prioritize and build against, because the guidance is
                > clear for everyone. It also gives guidance on budget available and
                > feasibility.

                Yes, it's good when it happens. But there are good things to do,
                even when it doesn't.

                Ron Jeffries
                www.XProgramming.com
                Analysis kills spontaneity.
                The grain once ground into flour germinates no more. -- Henri Amiel
              • Robin Dymond
                No question there are interesting things to do that an ROI is not required for, however, getting the customer to tell me do do something is not the problem,
                Message 7 of 12 , Jul 4, 2005
                • 0 Attachment
                  RE: [agile-usability] Re: Choice modeling and Agile?

                  No question there are interesting things to do that an ROI is not required for, however, getting the customer to tell me do 'do' something is not the problem, they are full of that. Lack of ROI analysis impacts project visibility, management buy-in, and budget availability. It make projects easier to kill, and introduces risk for all parties working on the project. Scope management is harder, because project owners are often not the users, or the system is being designed based on what we think we need. The key issues are budget CFO: "Why should I give you $2M if you can't show me an ROI?") and scope Client: "We must have features X,Y, Z, because it just 'won't work' if we don't".

                  Cheers,
                  Robin Dymond

                  -----Original Message-----
                  From: agile-usability@yahoogroups.com [mailto:agile-usability@yahoogroups.com] On Behalf Of Ron Jeffries
                  Sent: Monday, July 04, 2005 2:51 PM
                  To: agile-usability@yahoogroups.com
                  Subject: Re: [agile-usability] Re: Choice modeling and Agile?

                  On Monday, July 4, 2005, at 4:16:15 PM, Robin Dymond wrote:

                  > That valuation dilemma is exactly the problem I have on one current
                  > project. The client has a long list of features for an Intranet, their
                  > priorities, and a limited budget. They know the features are valuable
                  > to the users, but they don't know how valuable, therefore they don't
                  > know how to come up with an ROI.
                  > Even if they did come up with an ROI (a valuable exercise for dealing
                  > with CFOs) the assumptions in the ROI are WAGs (wild ass guesses). As
                  > with most projects, the wish list far exceeds the funding available.

                  Well, of course they "should" be able to express the ROI. But I'd suggest that it's not "necessary".

                  Given two features, if they can decide which one to do first, that's often "enough".

                  Sometimes folks have trouble doing that. I help them along by picking something obviously valuable and something obviously dull.

                  (There are always features that qualify.) Then I suggest that we do the dull one first, and defer the valuable one for a long time. They call me an idiot, and get about the business of deciding what to do next.

                   
                  > Another project has a very clear ROI: reduce call center calls by X %,
                  > and reduce costs per interaction by Y%. These are great metrics to
                  > prioritize and build against, because the guidance is clear for
                  > everyone. It also gives guidance on budget available and feasibility.

                  Yes, it's good when it happens. But there are good things to do, even when it doesn't.

                  Ron Jeffries
                  www.XProgramming.com
                  Analysis kills spontaneity.
                  The grain once ground into flour germinates no more. --  Henri Amiel



                   
                  Yahoo! Groups Links

                  <*> To visit your group on the web, go to:
                      http://groups.yahoo.com/group/agile-usability/

                  <*> To unsubscribe from this group, send an email to:
                      agile-usability-unsubscribe@yahoogroups.com

                  <*> Your use of Yahoo! Groups is subject to:
                      http://docs.yahoo.com/info/terms/
                   



                  The information contained in this message is confidential. It is intended to be read only by the individual or entity named above or their designee. If the reader of this message is not the intended recipient, you are hereby notified that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete or destroy any copy of this message.
                • Kevin Narey
                  ... A curious statement. Do you find that users of software, built using agile methods, enjoy it when they can t use the software because the architect of that
                  Message 8 of 12 , Jul 4, 2005
                  • 0 Attachment
                    Ron wrote:

                    >Analysis kills spontaneity

                    A curious statement.

                    Do you find that users of software, built using agile methods, enjoy
                    it when they can't use the software because the architect of that
                    software, as a result of not performing a prior phase of analysis (and
                    design) 'spontaneously' second guessed what their goals were? Can I
                    also ask what your views on innovation without analysis/research are?

                    //else

                    If you're looking for an approach to determining ROI you might want to
                    view Jared Spool's article:

                    http://www.uie.com/articles/cost_of_frustration/

                    Kevin
                  • Desilets, Alain
                    Well, of course they should be able to express the ROI. But I d suggest that it s not necessary . Given two features, if they can decide which one to do
                    Message 9 of 12 , Jul 4, 2005
                    • 0 Attachment
                      Well, of course they "should" be able to express the ROI. But I'd suggest that it's not "necessary".

                      Given two features, if they can decide which one to do first, that's often "enough".

                      Sometimes folks have trouble doing that. I help them along by picking something obviously valuable and something obviously dull. (There are always features that qualify.) Then I suggest that we do the dull one first, and defer the valuable one for a long time. They call me an idiot, and get about the business of deciding what to do next.

                      -- Alain:
                      Great suggestion Ron! I'll have to remember that.

                      Most people (including myself) can't quantify things in abolute terms. But they can usually rank things one against the other.
                      ----
                    • Jim Kauffman
                      That s one great thing about Agile, if it s being done right. Customers get lots of chances to change their priorities along the way. Jim K.
                      Message 10 of 12 , Jul 4, 2005
                      • 0 Attachment
                        That's one great thing about Agile, if it's being done right. Customers get
                        lots of chances to change their priorities along the way.


                        Jim K.


                        > -----Original Message-----
                        > From: agile-usability@yahoogroups.com
                        > [mailto:agile-usability@yahoogroups.com] On Behalf Of Desilets, Alain
                        > Sent: Monday, July 04, 2005 5:52 PM
                        > To: 'agile-usability@yahoogroups.com'
                        > Subject: RE: [agile-usability] Re: Choice modeling and Agile?
                        >
                        > Well, of course they "should" be able to express the ROI. But
                        > I'd suggest that it's not "necessary".
                        >
                        > Given two features, if they can decide which one to do first,
                        > that's often "enough".
                      • Ron Jeffries
                        ... Actually, Henri Amiel wrote that, and the complete quote, randomly ... It might be wise not to read too much into the workings of a random number
                        Message 11 of 12 , Jul 5, 2005
                        • 0 Attachment
                          On Monday, July 4, 2005, at 5:35:22 PM, Kevin Narey wrote:

                          > Ron wrote:

                          >>Analysis kills spontaneity

                          Actually, Henri Amiel wrote that, and the complete quote, randomly
                          selected by my email client, is:

                          >> Analysis kills spontaneity.
                          >> The grain once ground into flour germinates no more. -- Henri Amiel

                          > A curious statement.

                          It might be wise not to read too much into the workings of a random
                          number generator. It might be more fruitful to comment on what I
                          wrote, not on what Mr Amiel wrote. However, your questions below are
                          answerable. These are my answers: I don't know what Mr Amiel would
                          say, had he not unfortunately passed away in 1881.

                          > Do you find that users of software, built using agile methods, enjoy
                          > it when they can't use the software because the architect of that
                          > software, as a result of not performing a prior phase of analysis (and
                          > design) 'spontaneously' second guessed what their goals were?

                          Well, I find that that doesn't happen, and the form and tone of the
                          question makes me want to recommend a bit more study of agile
                          methods. Here are some reasons why:

                          Agile methods do analysis all the time, not just in a prior phase.

                          Agile teams work with the users of the software, addressing only
                          features that those users ask for, in the order they are
                          requested. The goals are explicit, discussed continuously.

                          The software is delivered frequently, every couple of weeks, or
                          every month, so that the customers can see it, use it, test it in
                          any way that they see fit.

                          Agile teams do design all the time, not just in a prior phase.

                          Agile teams start with a simple design at the beginning, enough to
                          support the few features they need to deliver at the beginning. In
                          order to sustain continuous delivery

                          Agile teams generally do not have an individual designated as "the
                          architect".

                          Agile teams generally share the conventional duties of development
                          teams, analysis, architecture, design, testing, coding, and so on.
                          Certainly each team will have individuals who are more or less
                          skilled in these areas, but it is rare to have specific
                          individuals called out into roles.

                          Agile teams value spontaneity, but also discipline.

                          Agility is about noticing what's going on and responding to it.
                          But spontaneity alone, the late Mr Amiel notwithstanding, does not
                          accomplish much. That's why Agile teams follow a discipline that
                          keeps them in touch, at all times, with the customer and what the
                          customer wants.

                          > Can I also ask what your views on innovation without
                          > analysis/research are?

                          Well, briefly, if you can imagine that, I think that innovation is
                          at its highest when one maintains a bit of distance from "reality",
                          but is quite aware of all that is going on. If by "analysis" we mean
                          "paying attention", if by "research" we mean "looking around", then
                          they're quite valuable to innovation.

                          If on the other hand we mean something more formal, rigid,
                          stratified, phased, then I would likely begin to come down more on
                          the side of the sadly departed Henri-Frederic.

                          Analysis and research, in the conventional sense, are likely to lead
                          to refinement. I would be less sanguine about their leading to true
                          innovation. But there's always a spectrum, a continuum: innovation
                          is very likely a soup made up of many ingredients.

                          > //else

                          > If you're looking for an approach to determining ROI you might want to
                          > view Jared Spool's article:

                          > http://www.uie.com/articles/cost_of_frustration/

                          It's an interesting article. As my original response said, at least
                          twice, I'm all for knowing ROI. I'm also pointing out that projects
                          can proceed on the basis of less detail, specifically users' ability
                          to choose between alternatives, even when they don't have detailed
                          numbers.

                          Ron Jeffries
                          www.XProgramming.com
                          Perhaps this Silver Bullet will tell you who I am ...
                        • grahamastles
                          While the emphasis on finding a dollar ROI for each feature may be an idea to make the customer more aware of the resources she is spending, I think that there
                          Message 12 of 12 , Jul 5, 2005
                          • 0 Attachment
                            While the emphasis on finding a dollar ROI for each feature may be an
                            idea to make the customer more aware of the resources she is spending,
                            I think that there is a risk to fall into the same legalistic trap
                            that traditional project management does when it treats estimates as
                            commitments and a schedule as a prediction.

                            It seems to me that the core purpose behind any prioritization is to
                            work on the most important things first. For this need, a ranking
                            gives you 90% of the value with 10% of the work.

                            Thus, I think that the discussion on "feature ROI" should be part of a
                            pep-talk in the initial project definition phase, with occasional
                            reminders during the planning game, but the tough work required to get
                            an actual ROI for each feature sounds non-agile in the sense that it
                            is "up-front work" that does not produce working code that the user
                            can actually approve. Since priority ranking is a cheaper alternative
                            that can get you to actual coding quicker, I would recommend keeping
                            feature-ROI as a conceptual construct and not a mathematical process.

                            In any case, if we go down this path, Agile would require having a
                            feedback process where you actually do testing to determine if your
                            calculations corresponded to reality, so that you could improve the
                            process on the next round. That sounds like a very long feedback loop
                            to me, and I wonder if the ROI is positive at all, and I certainly
                            think that most of our teams would find a higher ROI from implementing
                            or improving other aspects of our craft.
                          Your message has been successfully submitted and would be delivered to recipients shortly.