Loading ...
Sorry, an error occurred while loading the content.
 

Re: [agile-usability] Choice modeling and Agile?

Expand Messages
  • Adi The
    Jeff, Thanks for sharing the information about the choice modeling. One of our researchers conducted a similar workshop session called Idea Shopping after
    Message 1 of 12 , Jul 2, 2005
      Jeff,
      Thanks for sharing the information about the choice modeling.
      One of our researchers conducted a similar workshop session called
      "Idea Shopping" after
      preceding ethnographic field work results analysed and synthesised.
      For the "Idea Shopping" workshop, several UI paper prototypes, such as
      login UI prototypes, were prepared.
      Each prototype was assigned some "Kroner" values (Scandinavian
      currency) based on the analysis and synthesis of field work results
      and
      each workshop participant was given some "pocket money" to shop for
      the prototypes.
      The workshop participants could shop until they spent all their "money".
      During the shopping, the participants could provide instant feedback,
      ask questions, have dialogues with the other shoppers, etc.
      The results of the workshop were then used to narrow down the choices
      and refine the prototypes.

      --
      Adi B. Tedjasaputra

      IT and User Experience Manager
      TRANSLATE-EASY
      :: RFID and Human-centred Design Innovation Centre in Asia ::
      http://TRANSLATE-EASY.com

      Menara Kadin Indonesia 30th Floor
      Jl. Rasuna Said Block X-5, Kav 2-3
      Jakarta 12950
      INDONESIA

      Direct. +45 - 20 27 77 53
      Fax. +45 - 28 17 09 78
    • Lynn Miller
      In the poast we have used choice modeling during the product validation stage when we haven t got down to the details yet. Once down to the level of features,
      Message 2 of 12 , Jul 4, 2005
        In the poast we have used choice modeling during the product validation
        stage when we haven't got down to the details yet.

        Once down to the level of features, it often makes sense to use a
        similar but reversed method - you assign dollar values to the features
        and give the users a certain amount to spend.

        The nice thing about this second method is that you can balance the
        development costs of features by price. That is, a huge feature isn't
        given the same weight as a small feature because it costs more. So
        users can pick the big ones that are really important and mix in some of
        the smaller ones. The amount they have to spend depends on your release
        date so this works better with fixed release dates.

        I'm glad to see that you're combining the results with other data
        gathering methods. On its own this method doesn't produce full
        workflows - it just helps you to focus on what is important. And you
        have to have done the work to get the list down to a reasonable size
        before it goes in front of users.

        Lynn Miller

        Jeff Lash wrote:
        > I wanted to see if anyone was using choice modeling in conjunction
        > with Agile development?
        >
        > For those not familiar with it, choice modeling views the value of a
        > product (and the price one is willing to pay for it) as the sum of the
        > features and attributes the product contains. You ask participants not
        > just to rank elements in priority order, but instead to assign dollar
        > values to features based on their importance to the customer. The more
        > dollars assigned to a feature/attribute, the more important it is to
        > the customer.
        >
        > The resulting data can be used not only to understand what is
        > preferred, but how valuable each element is, how much more important
        > certain elements are than others, and potentially how much a customer
        > may be willing to pay for a product with a given set of features and
        > attributes.
        >
        > It's actually much more than that -- there's a good body of
        > statistical, econometric, and business academia work in the field --
        > but that's the simple, practical application of the idea.
        >
        > This seems to be a natural fit with the prioritization of features
        > that goes on for each iteration. I'm trying it out, combining the
        > results of choice modeling and traditional questionnaires (which
        > capture what people say they want) with data gathered during field
        > studies and ethnographic interviews (which captures what people really
        > need, often unspoken), and using all of that to better understand
        > prioritization. It's certainly a "lightweight" approach to choice
        > modeling, but just trying it out on a project, it seems to have
        > yielded some interesting insights.
        >
        > So, I wanted to see if anyone has tried anything similar, or has any
        > thoughts on this approach.
        >
        > Jeff
        >
        > jeff@...
      • Robin Dymond
        That valuation dilemma is exactly the problem I have on one current project. The client has a long list of features for an Intranet, their priorities, and a
        Message 3 of 12 , Jul 4, 2005
          That valuation dilemma is exactly the problem I have on one current project. The client has a long list of features for an Intranet, their priorities, and a limited budget. They know the features are valuable to the users, but they don't know how valuable, therefore they don't know how to come up with an ROI. Even if they did come up with an ROI (a valuable exercise for dealing with CFOs) the assumptions in the ROI are WAGs (wild ass guesses). As with most projects, the wish list far exceeds the funding available.
           
          Another project has a very clear ROI: reduce call center calls by X %, and reduce costs per interaction by Y%. These are great metrics to prioritize and build against, because the guidance is clear for everyone. It also gives guidance on budget available and feasibility.
           
          cheers,
          Robin Dymond

          From: agile-usability@yahoogroups.com [mailto:agile-usability@yahoogroups.com] On Behalf Of acockburn@...
          Sent: Saturday, July 02, 2005 10:15 AM
          To: agile-usability@yahoogroups.com
          Subject: [agile-usability] Re: Choice modeling and Agile?

          I've tried, but can't find anyone who can come up with the numbers.
          How do you get those detailed number?
          Alistair
           
          In a message dated 7/2/2005 9:02:02 A.M. Mountain Daylight Time, agile-usability@yahoogroups.com writes:
          Subject: Choice modeling and Agile?

          I wanted to see if anyone was using choice modeling in conjunction
          with Agile development?

          For those not familiar with it, choice modeling views the value of a
          product (and the price one is willing to pay for it) as the sum of the
          features and attributes the product contains. You ask participants not
          just to rank elements in priority order, but instead to assign dollar
          values to features based on their importance to the customer. The more
          dollars assigned to a feature/attribute, the more important it is to
          the customer.

          The resulting data can be used not only to understand what is
          preferred, but how valuable each element is, how much more important
          certain elements are than others, and potentially how much a customer
          may be willing to pay for a product with a given set of features and
          attributes.

          It's actually much more than that -- there's a good body of
          statistical, econometric, and business academia work in the field --
          but that's the simple, practical application of the idea.

          This seems to be a natural fit with the prioritization of features
          that goes on for each iteration. I'm trying it out, combining the
          results of choice modeling and traditional questionnaires (which
          capture what people say they want) with data gathered during field
          studies and ethnographic interviews (which captures what people really
          need, often unspoken), and using all of that to better understand
          prioritization. It's certainly a "lightweight" approach to choice
          modeling, but just trying it out on a project, it seems to have
          yielded some interesting insights.

          So, I wanted to see if anyone has tried anything similar, or has any
          thoughts on this approach.

          Jeff
           
          ==============================================
          Alistair Cockburn
          President, Humans and Technology

          801.582.3162
          1814 Ft Douglas Cir,
          Salt Lake City, UT 84103
          http://alistair.cockburn.us/
          acockburn@...
          ===============================

          "Surviving Object-Oriented Projects" (1998)
          "Writing Effective Use Cases" (Jolt Productivity Award 2001)
          "Agile Software Development" (Jolt Productivity Award 2002)
          "Crystal Clear: A Human-Powered Methodology for Small Teams" (Jolt Award Finalist 2004)

          "La perfection est atteinte non quand il ne reste rien a ajouter,
          mais quand il ne reste rien a enlever." (Saint-Exupery)

          "The first thing to build is trust." (Brad Appleton)
          ==============================================


          The information contained in this message is confidential. It is intended to be read only by the individual or entity named above or their designee. If the reader of this message is not the intended recipient, you are hereby notified that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete or destroy any copy of this message.
        • Ron Jeffries
          ... Well, of course they should be able to express the ROI. But I d suggest that it s not necessary . Given two features, if they can decide which one to do
          Message 4 of 12 , Jul 4, 2005
            On Monday, July 4, 2005, at 4:16:15 PM, Robin Dymond wrote:

            > That valuation dilemma is exactly the problem I have on one
            > current project. The client has a long list of features for an
            > Intranet, their priorities, and a limited budget. They know the
            > features are valuable to the users, but they don't know how
            > valuable, therefore they don't know how to come up with an ROI.
            > Even if they did come up with an ROI (a valuable exercise for
            > dealing with CFOs) the assumptions in the ROI are WAGs (wild ass
            > guesses). As with most projects, the wish list far exceeds the
            > funding available.

            Well, of course they "should" be able to express the ROI. But I'd
            suggest that it's not "necessary".

            Given two features, if they can decide which one to do first, that's
            often "enough".

            Sometimes folks have trouble doing that. I help them along by
            picking something obviously valuable and something obviously dull.
            (There are always features that qualify.) Then I suggest that we do
            the dull one first, and defer the valuable one for a long time. They
            call me an idiot, and get about the business of deciding what to do
            next.

            > Another project has a very clear ROI: reduce call center calls by
            > X %, and reduce costs per interaction by Y%. These are great
            > metrics to prioritize and build against, because the guidance is
            > clear for everyone. It also gives guidance on budget available and
            > feasibility.

            Yes, it's good when it happens. But there are good things to do,
            even when it doesn't.

            Ron Jeffries
            www.XProgramming.com
            Analysis kills spontaneity.
            The grain once ground into flour germinates no more. -- Henri Amiel
          • Robin Dymond
            No question there are interesting things to do that an ROI is not required for, however, getting the customer to tell me do do something is not the problem,
            Message 5 of 12 , Jul 4, 2005
              RE: [agile-usability] Re: Choice modeling and Agile?

              No question there are interesting things to do that an ROI is not required for, however, getting the customer to tell me do 'do' something is not the problem, they are full of that. Lack of ROI analysis impacts project visibility, management buy-in, and budget availability. It make projects easier to kill, and introduces risk for all parties working on the project. Scope management is harder, because project owners are often not the users, or the system is being designed based on what we think we need. The key issues are budget CFO: "Why should I give you $2M if you can't show me an ROI?") and scope Client: "We must have features X,Y, Z, because it just 'won't work' if we don't".

              Cheers,
              Robin Dymond

              -----Original Message-----
              From: agile-usability@yahoogroups.com [mailto:agile-usability@yahoogroups.com] On Behalf Of Ron Jeffries
              Sent: Monday, July 04, 2005 2:51 PM
              To: agile-usability@yahoogroups.com
              Subject: Re: [agile-usability] Re: Choice modeling and Agile?

              On Monday, July 4, 2005, at 4:16:15 PM, Robin Dymond wrote:

              > That valuation dilemma is exactly the problem I have on one current
              > project. The client has a long list of features for an Intranet, their
              > priorities, and a limited budget. They know the features are valuable
              > to the users, but they don't know how valuable, therefore they don't
              > know how to come up with an ROI.
              > Even if they did come up with an ROI (a valuable exercise for dealing
              > with CFOs) the assumptions in the ROI are WAGs (wild ass guesses). As
              > with most projects, the wish list far exceeds the funding available.

              Well, of course they "should" be able to express the ROI. But I'd suggest that it's not "necessary".

              Given two features, if they can decide which one to do first, that's often "enough".

              Sometimes folks have trouble doing that. I help them along by picking something obviously valuable and something obviously dull.

              (There are always features that qualify.) Then I suggest that we do the dull one first, and defer the valuable one for a long time. They call me an idiot, and get about the business of deciding what to do next.

               
              > Another project has a very clear ROI: reduce call center calls by X %,
              > and reduce costs per interaction by Y%. These are great metrics to
              > prioritize and build against, because the guidance is clear for
              > everyone. It also gives guidance on budget available and feasibility.

              Yes, it's good when it happens. But there are good things to do, even when it doesn't.

              Ron Jeffries
              www.XProgramming.com
              Analysis kills spontaneity.
              The grain once ground into flour germinates no more. --  Henri Amiel



               
              Yahoo! Groups Links

              <*> To visit your group on the web, go to:
                  http://groups.yahoo.com/group/agile-usability/

              <*> To unsubscribe from this group, send an email to:
                  agile-usability-unsubscribe@yahoogroups.com

              <*> Your use of Yahoo! Groups is subject to:
                  http://docs.yahoo.com/info/terms/
               



              The information contained in this message is confidential. It is intended to be read only by the individual or entity named above or their designee. If the reader of this message is not the intended recipient, you are hereby notified that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete or destroy any copy of this message.
            • Kevin Narey
              ... A curious statement. Do you find that users of software, built using agile methods, enjoy it when they can t use the software because the architect of that
              Message 6 of 12 , Jul 4, 2005
                Ron wrote:

                >Analysis kills spontaneity

                A curious statement.

                Do you find that users of software, built using agile methods, enjoy
                it when they can't use the software because the architect of that
                software, as a result of not performing a prior phase of analysis (and
                design) 'spontaneously' second guessed what their goals were? Can I
                also ask what your views on innovation without analysis/research are?

                //else

                If you're looking for an approach to determining ROI you might want to
                view Jared Spool's article:

                http://www.uie.com/articles/cost_of_frustration/

                Kevin
              • Desilets, Alain
                Well, of course they should be able to express the ROI. But I d suggest that it s not necessary . Given two features, if they can decide which one to do
                Message 7 of 12 , Jul 4, 2005
                  Well, of course they "should" be able to express the ROI. But I'd suggest that it's not "necessary".

                  Given two features, if they can decide which one to do first, that's often "enough".

                  Sometimes folks have trouble doing that. I help them along by picking something obviously valuable and something obviously dull. (There are always features that qualify.) Then I suggest that we do the dull one first, and defer the valuable one for a long time. They call me an idiot, and get about the business of deciding what to do next.

                  -- Alain:
                  Great suggestion Ron! I'll have to remember that.

                  Most people (including myself) can't quantify things in abolute terms. But they can usually rank things one against the other.
                  ----
                • Jim Kauffman
                  That s one great thing about Agile, if it s being done right. Customers get lots of chances to change their priorities along the way. Jim K.
                  Message 8 of 12 , Jul 4, 2005
                    That's one great thing about Agile, if it's being done right. Customers get
                    lots of chances to change their priorities along the way.


                    Jim K.


                    > -----Original Message-----
                    > From: agile-usability@yahoogroups.com
                    > [mailto:agile-usability@yahoogroups.com] On Behalf Of Desilets, Alain
                    > Sent: Monday, July 04, 2005 5:52 PM
                    > To: 'agile-usability@yahoogroups.com'
                    > Subject: RE: [agile-usability] Re: Choice modeling and Agile?
                    >
                    > Well, of course they "should" be able to express the ROI. But
                    > I'd suggest that it's not "necessary".
                    >
                    > Given two features, if they can decide which one to do first,
                    > that's often "enough".
                  • Ron Jeffries
                    ... Actually, Henri Amiel wrote that, and the complete quote, randomly ... It might be wise not to read too much into the workings of a random number
                    Message 9 of 12 , Jul 5, 2005
                      On Monday, July 4, 2005, at 5:35:22 PM, Kevin Narey wrote:

                      > Ron wrote:

                      >>Analysis kills spontaneity

                      Actually, Henri Amiel wrote that, and the complete quote, randomly
                      selected by my email client, is:

                      >> Analysis kills spontaneity.
                      >> The grain once ground into flour germinates no more. -- Henri Amiel

                      > A curious statement.

                      It might be wise not to read too much into the workings of a random
                      number generator. It might be more fruitful to comment on what I
                      wrote, not on what Mr Amiel wrote. However, your questions below are
                      answerable. These are my answers: I don't know what Mr Amiel would
                      say, had he not unfortunately passed away in 1881.

                      > Do you find that users of software, built using agile methods, enjoy
                      > it when they can't use the software because the architect of that
                      > software, as a result of not performing a prior phase of analysis (and
                      > design) 'spontaneously' second guessed what their goals were?

                      Well, I find that that doesn't happen, and the form and tone of the
                      question makes me want to recommend a bit more study of agile
                      methods. Here are some reasons why:

                      Agile methods do analysis all the time, not just in a prior phase.

                      Agile teams work with the users of the software, addressing only
                      features that those users ask for, in the order they are
                      requested. The goals are explicit, discussed continuously.

                      The software is delivered frequently, every couple of weeks, or
                      every month, so that the customers can see it, use it, test it in
                      any way that they see fit.

                      Agile teams do design all the time, not just in a prior phase.

                      Agile teams start with a simple design at the beginning, enough to
                      support the few features they need to deliver at the beginning. In
                      order to sustain continuous delivery

                      Agile teams generally do not have an individual designated as "the
                      architect".

                      Agile teams generally share the conventional duties of development
                      teams, analysis, architecture, design, testing, coding, and so on.
                      Certainly each team will have individuals who are more or less
                      skilled in these areas, but it is rare to have specific
                      individuals called out into roles.

                      Agile teams value spontaneity, but also discipline.

                      Agility is about noticing what's going on and responding to it.
                      But spontaneity alone, the late Mr Amiel notwithstanding, does not
                      accomplish much. That's why Agile teams follow a discipline that
                      keeps them in touch, at all times, with the customer and what the
                      customer wants.

                      > Can I also ask what your views on innovation without
                      > analysis/research are?

                      Well, briefly, if you can imagine that, I think that innovation is
                      at its highest when one maintains a bit of distance from "reality",
                      but is quite aware of all that is going on. If by "analysis" we mean
                      "paying attention", if by "research" we mean "looking around", then
                      they're quite valuable to innovation.

                      If on the other hand we mean something more formal, rigid,
                      stratified, phased, then I would likely begin to come down more on
                      the side of the sadly departed Henri-Frederic.

                      Analysis and research, in the conventional sense, are likely to lead
                      to refinement. I would be less sanguine about their leading to true
                      innovation. But there's always a spectrum, a continuum: innovation
                      is very likely a soup made up of many ingredients.

                      > //else

                      > If you're looking for an approach to determining ROI you might want to
                      > view Jared Spool's article:

                      > http://www.uie.com/articles/cost_of_frustration/

                      It's an interesting article. As my original response said, at least
                      twice, I'm all for knowing ROI. I'm also pointing out that projects
                      can proceed on the basis of less detail, specifically users' ability
                      to choose between alternatives, even when they don't have detailed
                      numbers.

                      Ron Jeffries
                      www.XProgramming.com
                      Perhaps this Silver Bullet will tell you who I am ...
                    • grahamastles
                      While the emphasis on finding a dollar ROI for each feature may be an idea to make the customer more aware of the resources she is spending, I think that there
                      Message 10 of 12 , Jul 5, 2005
                        While the emphasis on finding a dollar ROI for each feature may be an
                        idea to make the customer more aware of the resources she is spending,
                        I think that there is a risk to fall into the same legalistic trap
                        that traditional project management does when it treats estimates as
                        commitments and a schedule as a prediction.

                        It seems to me that the core purpose behind any prioritization is to
                        work on the most important things first. For this need, a ranking
                        gives you 90% of the value with 10% of the work.

                        Thus, I think that the discussion on "feature ROI" should be part of a
                        pep-talk in the initial project definition phase, with occasional
                        reminders during the planning game, but the tough work required to get
                        an actual ROI for each feature sounds non-agile in the sense that it
                        is "up-front work" that does not produce working code that the user
                        can actually approve. Since priority ranking is a cheaper alternative
                        that can get you to actual coding quicker, I would recommend keeping
                        feature-ROI as a conceptual construct and not a mathematical process.

                        In any case, if we go down this path, Agile would require having a
                        feedback process where you actually do testing to determine if your
                        calculations corresponded to reality, so that you could improve the
                        process on the next round. That sounds like a very long feedback loop
                        to me, and I wonder if the ROI is positive at all, and I certainly
                        think that most of our teams would find a higher ROI from implementing
                        or improving other aspects of our craft.
                      Your message has been successfully submitted and would be delivered to recipients shortly.