Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Agile UCD Velocity Points

Expand Messages
  • William Pietri
    Interesting question, Leina! How does your organization measure business value for the other stories in the queue? William
    Message 1 of 13 , May 8, 2008
    • 0 Attachment
      Interesting question, Leina!

      How does your organization measure business value for the other stories in the queue?

      William

      leina elgohari wrote:

      Thanks William

      That's a very valid point.

      What if the usability people wanted to dedicate an entire user story to doing usability stuff.

      From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

      Do you have any advice on working out the maths?

       

      Regards

      Leina
      --- On Thu, 5/8/08, William Pietri <william@...> wrote:

      From: William Pietri <william@...>
      Subject: Re: [agile-usability] Agile UCD Velocity Points
      To: agile-usability@yahoogroups.com
      Date: Thursday, May 8, 2008, 4:41 PM

      Leina wrote:
      > If the usability people are going to build in usability acceptance
      > criteria into the user stories it will no doubt add extra work.
      >
      > Do they need to estimate that work, work out their own velocity and
      > build it into the user story?
      >

      One of the things I see teams struggle with during agile adoption is
      going from a team with formal roles to a more collaborative team.

      In the beginning, when people are role oriented, database work is done
      by the database guys, back-end development is done by the server-side
      specialists, front-end development is done by the web people, and
      usability is done by the human factors expert. I think this is a natural
      place to start, but should be looked at as something to move away from
      over time.

      I think each estimated story should have a single estimate that
      represents the whole team's estimate, including everything it takes to
      get to 100% done. However, when scheduling stories within iterations,
      the team should be sensitive to individual overload and burnout.

      During iterations, teams should be working together in ways that provide
      some cross-training. If not enough of that is happening, explicitly
      schedule it in the course of working on stories. E.g., get together with
      the front-end people, and work out the UI together. It will take a
      while, but eventually your team will pick up some usability skills,
      allowing the basic stuff to be done by a broad range of people. Then the
      team can adjust organically to stories with different workloads, making
      the concept of a team velocity much more real.

      William


    • leina elgohari
      The Client would supply the figures. So an example (developer) user story reads like so: As a trainer I want to present training courses So that Company Z can
      Message 2 of 13 , May 8, 2008
      • 0 Attachment

        The Client would supply the figures. So an example (developer) user story reads like so:

         

        As a trainer

        I want to present training courses

        So that Company Z can improve resource utilisation by 10% (=£8K)

        If size = 8

        Then difficulty rating =  business value / size = 100

         

         

        The difficulty arises when you decide to write the user story from a usability angle:

         

        As a trainer

        I want to present quality training courses (quality here means providing an enjoyable user experience)

        How do you measure the benefits to Company Z. In other words how do you put a value on enjoyment/user satisfaction?

         

        Regards

        Leina

        --- On Thu, 5/8/08, William Pietri <william@...> wrote:

        From: William Pietri <william@...>
        Subject: Re: [agile-usability] Agile UCD Velocity Points
        To: agile-usability@yahoogroups.com
        Date: Thursday, May 8, 2008, 11:08 PM

        Interesting question, Leina!

        How does your organization measure business value for the other stories in the queue?

        William

        leina elgohari wrote:

        Thanks William

        That's a very valid point.

        What if the usability people wanted to dedicate an entire user story to doing usability stuff.

        From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

        Do you have any advice on working out the maths?

         

        Regards

        Leina
        --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

        From: William Pietri <william@scissor. com>
        Subject: Re: [agile-usability] Agile UCD Velocity Points
        To: agile-usability@ yahoogroups. com
        Date: Thursday, May 8, 2008, 4:41 PM

        Leina wrote:
        > If the usability people are going to build in usability acceptance
        > criteria into the user stories it will no doubt add extra work.
        >
        > Do they need to estimate that work, work out their own velocity and
        > build it into the user story?
        >

        One of the things I see teams struggle with during agile adoption is
        going from a team with formal roles to a more collaborative team.

        In the beginning, when people are role oriented, database work is done
        by the database guys, back-end development is done by the server-side
        specialists, front-end development is done by the web people, and
        usability is done by the human factors expert. I think this is a natural
        place to start, but should be looked at as something to move away from
        over time.

        I think each estimated story should have a single estimate that
        represents the whole team's estimate, including everything it takes to
        get to 100% done. However, when scheduling stories within iterations,
        the team should be sensitive to individual overload and burnout.

        During iterations, teams should be working together in ways that provide
        some cross-training. If not enough of that is happening, explicitly
        schedule it in the course of working on stories. E.g., get together with
        the front-end people, and work out the UI together. It will take a
        while, but eventually your team will pick up some usability skills,
        allowing the basic stuff to be done by a broad range of people. Then the
        team can adjust organically to stories with different workloads, making
        the concept of a team velocity much more real.

        William


      • William Pietri
        ... Well, Leina, most of the teams I coach judge value intuitively or with proximate metrics, so take this with a grain of salt. If I had to justify usability
        Message 3 of 13 , May 8, 2008
        • 0 Attachment

          Leina wrote:
          What if the usability people wanted to dedicate an entire user story to doing usability stuff.

          From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

          Do you have any advice on working out the maths?


          Well, Leina, most of the teams I coach judge value intuitively or with proximate metrics, so take this with a grain of salt.

          If I had to justify usability in that framework, I'd start by finding statistics on the kinds of improvement you get out of usability improvements. Next I'd try to get some statistics on the actual use of my app. Then I'd try to build a mathematical model of a particular flow and calculate value for usability improvements.

          For example, suppose you're at Amazon, and you suspect people are getting confused during the checkout process and are abandoning their carts.

          Step one would be to find out what sort of improvements you might see in improved usability. Let's say you decide that particular fixes to increase clarity could keep people from giving up 5% of the time.

          Then you'd look at usage statistics. Suppose that the number of abandoned carts is 1000/day, with $100 of merch in each cart, for a total of $100k left in carts.

          Next you'd do look at the steps in the flow and see what you could do. You could apply these techniques at two different stages, giving you a total of 10% reduction in lost carts, saving $10k a day, or $3.65 million annually.


          For these kinds of discussions, you hopefully can lean a lot on your client for this. They already should have a clear model of their business. It's worth just asking them how much increased customer satisfaction or reduced search time or whatever would be worth to them.

          To read more, I'd start with Jakob Nielsen's stuff on this:

          http://www.useit.com/alertbox/roi.html
          http://www.useit.com/alertbox/roi-first-study.html
          http://www.useit.com/alertbox/intranet-usability.html

          And I'm sure others can suggest more material along these lines.

          My main tip, though, is to start learning to put things in business terms. Quality and enjoyment might seem a little fuzzy to some suit-wearers, but when you talk about increased customer retention, improved referrals, more purchases, or less wasted staff time, they'll pay much better attention.

          Best,

          William


          leina elgohari wrote:

          The Client would supply the figures. So an example (developer) user story reads like so:

           

          As a trainer

          I want to present training courses

          So that Company Z can improve resource utilisation by 10% (=£8K)

          If size = 8

          Then difficulty rating =  business value / size = 100

           

           

          The difficulty arises when you decide to write the user story from a usability angle:

           

          As a trainer

          I want to present quality training courses (quality here means providing an enjoyable user experience)

          How do you measure the benefits to Company Z. In other words how do you put a value on enjoyment/user satisfaction?

           

          Regards

          Leina

          --- On Thu, 5/8/08, William Pietri <william@...> wrote:

          From: William Pietri <william@...>
          Subject: Re: [agile-usability] Agile UCD Velocity Points
          To: agile-usability@yahoogroups.com
          Date: Thursday, May 8, 2008, 11:08 PM

          Interesting question, Leina!

          How does your organization measure business value for the other stories in the queue?

          William

          leina elgohari wrote:

          Thanks William

          That's a very valid point.

          What if the usability people wanted to dedicate an entire user story to doing usability stuff.

          From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

          Do you have any advice on working out the maths?

           

          Regards

          Leina
          --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

          From: William Pietri <william@scissor. com>
          Subject: Re: [agile-usability] Agile UCD Velocity Points
          To: agile-usability@ yahoogroups. com
          Date: Thursday, May 8, 2008, 4:41 PM

          Leina wrote:
          > If the usability people are going to build in usability acceptance
          > criteria into the user stories it will no doubt add extra work.
          >
          > Do they need to estimate that work, work out their own velocity and
          > build it into the user story?
          >

          One of the things I see teams struggle with during agile adoption is
          going from a team with formal roles to a more collaborative team.

          In the beginning, when people are role oriented, database work is done
          by the database guys, back-end development is done by the server-side
          specialists, front-end development is done by the web people, and
          usability is done by the human factors expert. I think this is a natural
          place to start, but should be looked at as something to move away from
          over time.

          I think each estimated story should have a single estimate that
          represents the whole team's estimate, including everything it takes to
          get to 100% done. However, when scheduling stories within iterations,
          the team should be sensitive to individual overload and burnout.

          During iterations, teams should be working together in ways that provide
          some cross-training. If not enough of that is happening, explicitly
          schedule it in the course of working on stories. E.g., get together with
          the front-end people, and work out the UI together. It will take a
          while, but eventually your team will pick up some usability skills,
          allowing the basic stuff to be done by a broad range of people. Then the
          team can adjust organically to stories with different workloads, making
          the concept of a team velocity much more real.

          William



        • Ron Jeffries
          ... Customer determines stories based on (subjective) value. If they want a usability story, they ll schedule it. If they don t, they won t. Does that seem OK?
          Message 4 of 13 , May 8, 2008
          • 0 Attachment
            Hello, leina. On Thursday, May 8, 2008, at 5:49:06 PM, you wrote:

            > What if the usability people wanted to dedicate an entire user story to doing usability stuff.

            > From where would they get the business value given that its not a
            > straightforward process to be able to quantify the business value where usability is concerned?

            > Do you have any advice on working out the maths?

            Customer determines stories based on (subjective) value. If they
            want a usability story, they'll schedule it. If they don't, they
            won't. Does that seem OK?

            Ron Jeffries
            www.XProgramming.com
            Adapt, improvise, overcome.
            --Gunnery Sergeant Tom Highway (Heartbreak Ridge)
          • leina elgohari
            I really wanted to keep the maths very very simple (ROI can make the maths complicated). I was toying around with Mynott C et al (1994) table showing the cost
            Message 5 of 13 , May 9, 2008
            • 0 Attachment

              I really wanted to keep the maths very very simple (ROI can make the maths complicated). I was toying around with  Mynott C et al (1994) table showing the cost of making changes:

              Concept – 1

              Detail design – 10

              Tooling – 100

              Testing – 1000

              Post-Release – 10000

              I thought to use the above, invert the figures in the table so that at:

              Sprint/Release 1 – use 10000 to calculate the ‘simple maths’(this will give a high number thus making it imperative to carry out the usability user story(s) in this release)

              Sprint/Release 2 – use 1000 to calculate the ‘simple maths’

              Sprint/Release 3 – use 100  to caculate the 'simple maths'

              and so on…


              However, there is a problem:

              (1) My calculations assume that usability involvement peters out towards the tail-end of the project. Is that case with usability in an agile project?

              (2) Mynott's table refers to cost of making changes where the project has adopted a Waterfall approach. Is Mynott's table relevent in agile projects?



              --- On Fri, 5/9/08, William Pietri <william@...> wrote:
              From: William Pietri <william@...>
              Subject: Re: [agile-usability] Agile UCD Velocity Points
              To: agile-usability@yahoogroups.com
              Date: Friday, May 9, 2008, 2:14 AM


              Leina wrote:

              What if the usability people wanted to dedicate an entire user story to doing usability stuff.

              From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

              Do you have any advice on working out the maths?


              Well, Leina, most of the teams I coach judge value intuitively or with proximate metrics, so take this with a grain of salt.

              If I had to justify usability in that framework, I'd start by finding statistics on the kinds of improvement you get out of usability improvements. Next I'd try to get some statistics on the actual use of my app. Then I'd try to build a mathematical model of a particular flow and calculate value for usability improvements.

              For example, suppose you're at Amazon, and you suspect people are getting confused during the checkout process and are abandoning their carts.

              Step one would be to find out what sort of improvements you might see in improved usability. Let's say you decide that particular fixes to increase clarity could keep people from giving up 5% of the time.

              Then you'd look at usage statistics. Suppose that the number of abandoned carts is 1000/day, with $100 of merch in each cart, for a total of $100k left in carts.

              Next you'd do look at the steps in the flow and see what you could do. You could apply these techniques at two different stages, giving you a total of 10% reduction in lost carts, saving $10k a day, or $3.65 million annually.


              For these kinds of discussions, you hopefully can lean a lot on your client for this. They already should have a clear model of their business. It's worth just asking them how much increased customer satisfaction or reduced search time or whatever would be worth to them.

              To read more, I'd start with Jakob Nielsen's stuff on this:

              http://www.useit. com/alertbox/ roi.html
              http://www.useit. com/alertbox/ roi-first- study.html
              http://www.useit. com/alertbox/ intranet- usability. html

              And I'm sure others can suggest more material along these lines.

              My main tip, though, is to start learning to put things in business terms. Quality and enjoyment might seem a little fuzzy to some suit-wearers, but when you talk about increased customer retention, improved referrals, more purchases, or less wasted staff time, they'll pay much better attention.

              Best,

              William


              leina elgohari wrote:

              The Client would supply the figures. So an example (developer) user story reads like so:

               

              As a trainer

              I want to present training courses

              So that Company Z can improve resource utilisation by 10% (=£8K)

              If size = 8

              Then difficulty rating =  business value / size = 100

               

               

              The difficulty arises when you decide to write the user story from a usability angle:

               

              As a trainer

              I want to present quality training courses (quality here means providing an enjoyable user experience)

              How do you measure the benefits to Company Z. In other words how do you put a value on enjoyment/user satisfaction?

               

              Regards

              Leina

              --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

              From: William Pietri <william@scissor. com>
              Subject: Re: [agile-usability] Agile UCD Velocity Points
              To: agile-usability@ yahoogroups. com
              Date: Thursday, May 8, 2008, 11:08 PM

              Interesting question, Leina!

              How does your organization measure business value for the other stories in the queue?

              William

              leina elgohari wrote:

              Thanks William

              That's a very valid point.

              What if the usability people wanted to dedicate an entire user story to doing usability stuff.

              From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

              Do you have any advice on working out the maths?

               

              Regards

              Leina
              --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

              From: William Pietri <william@scissor. com>
              Subject: Re: [agile-usability] Agile UCD Velocity Points
              To: agile-usability@ yahoogroups. com
              Date: Thursday, May 8, 2008, 4:41 PM

              Leina wrote:
              > If the usability people are going to build in usability acceptance
              > criteria into the user stories it will no doubt add extra work.
              >
              > Do they need to estimate that work, work out their own velocity and
              > build it into the user story?
              >

              One of the things I see teams struggle with during agile adoption is
              going from a team with formal roles to a more collaborative team.

              In the beginning, when people are role oriented, database work is done
              by the database guys, back-end development is done by the server-side
              specialists, front-end development is done by the web people, and
              usability is done by the human factors expert. I think this is a natural
              place to start, but should be looked at as something to move away from
              over time.

              I think each estimated story should have a single estimate that
              represents the whole team's estimate, including everything it takes to
              get to 100% done. However, when scheduling stories within iterations,
              the team should be sensitive to individual overload and burnout.

              During iterations, teams should be working together in ways that provide
              some cross-training. If not enough of that is happening, explicitly
              schedule it in the course of working on stories. E.g., get together with
              the front-end people, and work out the UI together. It will take a
              while, but eventually your team will pick up some usability skills,
              allowing the basic stuff to be done by a broad range of people. Then the
              team can adjust organically to stories with different workloads, making
              the concept of a team velocity much more real.

              William



            • meszaros_susan
              Looks pretty complicated to me...I don t understand the terminology (without seriously considering and reading it), so why would the customer understand? I
              Message 6 of 13 , May 9, 2008
              • 0 Attachment
                Looks pretty complicated to me...I don't understand the terminology
                (without seriously considering and reading it), so why would the
                customer understand?

                I think you have to speak their language, and as William pointed out a
                simple, quick and dirty calculation on the improvement in revenue due
                to a small change as a result of usability improvements would
                potentially scream "fix me fix me" to the cheque writers. Make it
                meaningful to your target and they will not only understand but act.
                Make them work to understand and you'll lose them (gee, isn't this
                what happens with a bad interface or poor usability?)

                susan
              • William Pietri
                ... Yes. This is why most teams I work with judge the value of features intuitively or through proximate measures, like user activity. The only people I have
                Message 7 of 13 , May 9, 2008
                • 0 Attachment

                  leina elgohari wrote:
                  I really wanted to keep the maths very very simple (ROI can make the maths complicated)

                  Yes. This is why most teams I work with judge the value of features intuitively or through proximate measures, like user activity.

                  The only people I have seen successfully use rigorous money-based metrics for business value are large companies with relatively stable business models and one or more people devoted full time to tracking metrics and modeling money flow. And there I think I've only seen it used at the project level, not the story level.

                  I imagine you could also use money-based metrics in a casual way, as a way for people try to quantify their intuitions a little. I think that would only work if your business model is pretty stable, though.

                  (1) My calculations assume that usability involvement peters out towards the tail-end of the project. Is that case with usability in an agile project?


                  Not in my experience. Many of the projects I work on don't end, so I don't have a ton of data for you, but I generally see usability involvement as relatively stable. I see more usability-focused or usability-generated stories later in a project, driven from real-world data.

                  For example, just this week I saw a team release a usability-improving feature to reduce people dropping out in the middle of a flow. And now they're working on a feature to give them better metrics that they will then use to drive future usability decisions.

                  (2) Mynott's table refers to cost of making changes where the project has adopted a Waterfall approach. Is Mynott's table relevent in agile projects?

                  If it is, then we have failed, and should go back to waterfall.

                  My belief is that the cost-of-change curve has been dramatically lowered, and that other cost curves (e.g., increasing cost of distant research and planning, increasing chance of external change, increasing chance of chained errors, cost of money spent with no return) can more than compensate for the rest.

                  At least in my on-the-web world, "ship it and see what happens" isn't just a viable alternative, it's seen as superior in many cases, especially when using techniques like A/B testing.  So I think you should avoid using a mathematical model that encourages a lot of work based on speculation.


                  Hoping that helps,

                  William
                • leina elgohari
                  Hi Susan Thanks for your advice. If it looks too complicated then... well I m not going to ditch it all together but need to rethink it through. But I ll take
                  Message 8 of 13 , May 12, 2008
                  • 0 Attachment

                    Hi Susan

                    Thanks for your advice.

                    If it looks too complicated then... well I'm not going to ditch it all together but need to rethink it through.

                    But I'll take on board your advice. I thought you had to come up with something precise and concise but the message I'm getting is provide a rough guesstimate of the figures.

                    Leina

                     



                    --- On Fri, 5/9/08, meszaros_susan <susan.meszaros@...> wrote:

                    From: meszaros_susan <susan.meszaros@...>
                    Subject: [agile-usability] Re: Agile UCD Velocity Points
                    To: agile-usability@yahoogroups.com
                    Date: Friday, May 9, 2008, 10:57 AM

                    Looks pretty complicated to me...I don't understand the terminology
                    (without seriously considering and reading it), so why would the
                    customer understand?

                    I think you have to speak their language, and as William pointed out a
                    simple, quick and dirty calculation on the improvement in revenue due
                    to a small change as a result of usability improvements would
                    potentially scream "fix me fix me" to the cheque writers. Make it
                    meaningful to your target and they will not only understand but act.
                    Make them work to understand and you'll lose them (gee, isn't this
                    what happens with a bad interface or poor usability?)

                    susan

                  • Ron Jeffries
                    ... The nature of the problem, estimating velocity, and tracking it, is such that precision is essentially impossible. Imagine estimating your velocity driving
                    Message 9 of 13 , May 12, 2008
                    • 0 Attachment
                      Hello, leina. On Monday, May 12, 2008, at 5:22:46 AM, you wrote:

                      > Thanks for your advice.

                      > If it looks too complicated then... well I'm not going to ditch
                      > it all together but need to rethink it through.

                      > But I'll take on board your advice. I thought you had to come up
                      > with something precise and concise but the message I'm getting is
                      > provide a rough guesstimate of the figures.

                      The nature of the problem, estimating velocity, and tracking it, is
                      such that precision is essentially impossible.

                      Imagine estimating your velocity driving across country. Estimating
                      to even the nearest mile per hour over each short interval is simply
                      silly. Tracking our speed over small intervals won't work because of
                      traffic, weather, and other factors. Keeping track of our overall
                      speed across country is interesting, but precision would be entirely
                      wasted: one extra rest stop or a slow waitress at lunch will change
                      the number substantially.

                      We find that estimating stories in small proportional integers 1, 2,
                      3, and counting the points upon completion, is sufficient for most
                      purposes.

                      For a sense of the level of precision that I think is appropriate
                      for most projects, you might enjoy this article:

                      Big Visible Charts

                      It's time to revisit the topic of Big Visible Charts. Display
                      important project information not in some formal way, not on the
                      web, not in PowerPoint, but in charts on the wall that no one can
                      miss. [Updated: Velocity Charts]

                      http://www.xprogramming.com/xpmag/BigVisibleCharts.htm

                      Regards,

                      Ron Jeffries
                      www.XProgramming.com
                      There's a difference between righteous anger and just being crabby.
                      --Barbara Richmond
                    Your message has been successfully submitted and would be delivered to recipients shortly.