Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Agile UCD Velocity Points

Expand Messages
  • leina elgohari
    Thanks William That s a very valid point. What if the usability people wanted to dedicate an entire user story to doing usability stuff. From where would they
    Message 1 of 13 , May 8 2:49 PM
    • 0 Attachment

      Thanks William

      That's a very valid point.

      What if the usability people wanted to dedicate an entire user story to doing usability stuff.

      From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

      Do you have any advice on working out the maths?

       

      Regards

      Leina

      --- On Thu, 5/8/08, William Pietri <william@...> wrote:

      From: William Pietri <william@...>
      Subject: Re: [agile-usability] Agile UCD Velocity Points
      To: agile-usability@yahoogroups.com
      Date: Thursday, May 8, 2008, 4:41 PM

      Leina wrote:
      > If the usability people are going to build in usability acceptance
      > criteria into the user stories it will no doubt add extra work.
      >
      > Do they need to estimate that work, work out their own velocity and
      > build it into the user story?
      >

      One of the things I see teams struggle with during agile adoption is
      going from a team with formal roles to a more collaborative team.

      In the beginning, when people are role oriented, database work is done
      by the database guys, back-end development is done by the server-side
      specialists, front-end development is done by the web people, and
      usability is done by the human factors expert. I think this is a natural
      place to start, but should be looked at as something to move away from
      over time.

      I think each estimated story should have a single estimate that
      represents the whole team's estimate, including everything it takes to
      get to 100% done. However, when scheduling stories within iterations,
      the team should be sensitive to individual overload and burnout.

      During iterations, teams should be working together in ways that provide
      some cross-training. If not enough of that is happening, explicitly
      schedule it in the course of working on stories. E.g., get together with
      the front-end people, and work out the UI together. It will take a
      while, but eventually your team will pick up some usability skills,
      allowing the basic stuff to be done by a broad range of people. Then the
      team can adjust organically to stories with different workloads, making
      the concept of a team velocity much more real.

      William

    • William Pietri
      Interesting question, Leina! How does your organization measure business value for the other stories in the queue? William
      Message 2 of 13 , May 8 3:08 PM
      • 0 Attachment
        Interesting question, Leina!

        How does your organization measure business value for the other stories in the queue?

        William

        leina elgohari wrote:

        Thanks William

        That's a very valid point.

        What if the usability people wanted to dedicate an entire user story to doing usability stuff.

        From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

        Do you have any advice on working out the maths?

         

        Regards

        Leina
        --- On Thu, 5/8/08, William Pietri <william@...> wrote:

        From: William Pietri <william@...>
        Subject: Re: [agile-usability] Agile UCD Velocity Points
        To: agile-usability@yahoogroups.com
        Date: Thursday, May 8, 2008, 4:41 PM

        Leina wrote:
        > If the usability people are going to build in usability acceptance
        > criteria into the user stories it will no doubt add extra work.
        >
        > Do they need to estimate that work, work out their own velocity and
        > build it into the user story?
        >

        One of the things I see teams struggle with during agile adoption is
        going from a team with formal roles to a more collaborative team.

        In the beginning, when people are role oriented, database work is done
        by the database guys, back-end development is done by the server-side
        specialists, front-end development is done by the web people, and
        usability is done by the human factors expert. I think this is a natural
        place to start, but should be looked at as something to move away from
        over time.

        I think each estimated story should have a single estimate that
        represents the whole team's estimate, including everything it takes to
        get to 100% done. However, when scheduling stories within iterations,
        the team should be sensitive to individual overload and burnout.

        During iterations, teams should be working together in ways that provide
        some cross-training. If not enough of that is happening, explicitly
        schedule it in the course of working on stories. E.g., get together with
        the front-end people, and work out the UI together. It will take a
        while, but eventually your team will pick up some usability skills,
        allowing the basic stuff to be done by a broad range of people. Then the
        team can adjust organically to stories with different workloads, making
        the concept of a team velocity much more real.

        William


      • leina elgohari
        The Client would supply the figures. So an example (developer) user story reads like so: As a trainer I want to present training courses So that Company Z can
        Message 3 of 13 , May 8 3:41 PM
        • 0 Attachment

          The Client would supply the figures. So an example (developer) user story reads like so:

           

          As a trainer

          I want to present training courses

          So that Company Z can improve resource utilisation by 10% (=£8K)

          If size = 8

          Then difficulty rating =  business value / size = 100

           

           

          The difficulty arises when you decide to write the user story from a usability angle:

           

          As a trainer

          I want to present quality training courses (quality here means providing an enjoyable user experience)

          How do you measure the benefits to Company Z. In other words how do you put a value on enjoyment/user satisfaction?

           

          Regards

          Leina

          --- On Thu, 5/8/08, William Pietri <william@...> wrote:

          From: William Pietri <william@...>
          Subject: Re: [agile-usability] Agile UCD Velocity Points
          To: agile-usability@yahoogroups.com
          Date: Thursday, May 8, 2008, 11:08 PM

          Interesting question, Leina!

          How does your organization measure business value for the other stories in the queue?

          William

          leina elgohari wrote:

          Thanks William

          That's a very valid point.

          What if the usability people wanted to dedicate an entire user story to doing usability stuff.

          From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

          Do you have any advice on working out the maths?

           

          Regards

          Leina
          --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

          From: William Pietri <william@scissor. com>
          Subject: Re: [agile-usability] Agile UCD Velocity Points
          To: agile-usability@ yahoogroups. com
          Date: Thursday, May 8, 2008, 4:41 PM

          Leina wrote:
          > If the usability people are going to build in usability acceptance
          > criteria into the user stories it will no doubt add extra work.
          >
          > Do they need to estimate that work, work out their own velocity and
          > build it into the user story?
          >

          One of the things I see teams struggle with during agile adoption is
          going from a team with formal roles to a more collaborative team.

          In the beginning, when people are role oriented, database work is done
          by the database guys, back-end development is done by the server-side
          specialists, front-end development is done by the web people, and
          usability is done by the human factors expert. I think this is a natural
          place to start, but should be looked at as something to move away from
          over time.

          I think each estimated story should have a single estimate that
          represents the whole team's estimate, including everything it takes to
          get to 100% done. However, when scheduling stories within iterations,
          the team should be sensitive to individual overload and burnout.

          During iterations, teams should be working together in ways that provide
          some cross-training. If not enough of that is happening, explicitly
          schedule it in the course of working on stories. E.g., get together with
          the front-end people, and work out the UI together. It will take a
          while, but eventually your team will pick up some usability skills,
          allowing the basic stuff to be done by a broad range of people. Then the
          team can adjust organically to stories with different workloads, making
          the concept of a team velocity much more real.

          William


        • William Pietri
          ... Well, Leina, most of the teams I coach judge value intuitively or with proximate metrics, so take this with a grain of salt. If I had to justify usability
          Message 4 of 13 , May 8 6:14 PM
          • 0 Attachment

            Leina wrote:
            What if the usability people wanted to dedicate an entire user story to doing usability stuff.

            From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

            Do you have any advice on working out the maths?


            Well, Leina, most of the teams I coach judge value intuitively or with proximate metrics, so take this with a grain of salt.

            If I had to justify usability in that framework, I'd start by finding statistics on the kinds of improvement you get out of usability improvements. Next I'd try to get some statistics on the actual use of my app. Then I'd try to build a mathematical model of a particular flow and calculate value for usability improvements.

            For example, suppose you're at Amazon, and you suspect people are getting confused during the checkout process and are abandoning their carts.

            Step one would be to find out what sort of improvements you might see in improved usability. Let's say you decide that particular fixes to increase clarity could keep people from giving up 5% of the time.

            Then you'd look at usage statistics. Suppose that the number of abandoned carts is 1000/day, with $100 of merch in each cart, for a total of $100k left in carts.

            Next you'd do look at the steps in the flow and see what you could do. You could apply these techniques at two different stages, giving you a total of 10% reduction in lost carts, saving $10k a day, or $3.65 million annually.


            For these kinds of discussions, you hopefully can lean a lot on your client for this. They already should have a clear model of their business. It's worth just asking them how much increased customer satisfaction or reduced search time or whatever would be worth to them.

            To read more, I'd start with Jakob Nielsen's stuff on this:

            http://www.useit.com/alertbox/roi.html
            http://www.useit.com/alertbox/roi-first-study.html
            http://www.useit.com/alertbox/intranet-usability.html

            And I'm sure others can suggest more material along these lines.

            My main tip, though, is to start learning to put things in business terms. Quality and enjoyment might seem a little fuzzy to some suit-wearers, but when you talk about increased customer retention, improved referrals, more purchases, or less wasted staff time, they'll pay much better attention.

            Best,

            William


            leina elgohari wrote:

            The Client would supply the figures. So an example (developer) user story reads like so:

             

            As a trainer

            I want to present training courses

            So that Company Z can improve resource utilisation by 10% (=£8K)

            If size = 8

            Then difficulty rating =  business value / size = 100

             

             

            The difficulty arises when you decide to write the user story from a usability angle:

             

            As a trainer

            I want to present quality training courses (quality here means providing an enjoyable user experience)

            How do you measure the benefits to Company Z. In other words how do you put a value on enjoyment/user satisfaction?

             

            Regards

            Leina

            --- On Thu, 5/8/08, William Pietri <william@...> wrote:

            From: William Pietri <william@...>
            Subject: Re: [agile-usability] Agile UCD Velocity Points
            To: agile-usability@yahoogroups.com
            Date: Thursday, May 8, 2008, 11:08 PM

            Interesting question, Leina!

            How does your organization measure business value for the other stories in the queue?

            William

            leina elgohari wrote:

            Thanks William

            That's a very valid point.

            What if the usability people wanted to dedicate an entire user story to doing usability stuff.

            From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

            Do you have any advice on working out the maths?

             

            Regards

            Leina
            --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

            From: William Pietri <william@scissor. com>
            Subject: Re: [agile-usability] Agile UCD Velocity Points
            To: agile-usability@ yahoogroups. com
            Date: Thursday, May 8, 2008, 4:41 PM

            Leina wrote:
            > If the usability people are going to build in usability acceptance
            > criteria into the user stories it will no doubt add extra work.
            >
            > Do they need to estimate that work, work out their own velocity and
            > build it into the user story?
            >

            One of the things I see teams struggle with during agile adoption is
            going from a team with formal roles to a more collaborative team.

            In the beginning, when people are role oriented, database work is done
            by the database guys, back-end development is done by the server-side
            specialists, front-end development is done by the web people, and
            usability is done by the human factors expert. I think this is a natural
            place to start, but should be looked at as something to move away from
            over time.

            I think each estimated story should have a single estimate that
            represents the whole team's estimate, including everything it takes to
            get to 100% done. However, when scheduling stories within iterations,
            the team should be sensitive to individual overload and burnout.

            During iterations, teams should be working together in ways that provide
            some cross-training. If not enough of that is happening, explicitly
            schedule it in the course of working on stories. E.g., get together with
            the front-end people, and work out the UI together. It will take a
            while, but eventually your team will pick up some usability skills,
            allowing the basic stuff to be done by a broad range of people. Then the
            team can adjust organically to stories with different workloads, making
            the concept of a team velocity much more real.

            William



          • Ron Jeffries
            ... Customer determines stories based on (subjective) value. If they want a usability story, they ll schedule it. If they don t, they won t. Does that seem OK?
            Message 5 of 13 , May 8 7:00 PM
            • 0 Attachment
              Hello, leina. On Thursday, May 8, 2008, at 5:49:06 PM, you wrote:

              > What if the usability people wanted to dedicate an entire user story to doing usability stuff.

              > From where would they get the business value given that its not a
              > straightforward process to be able to quantify the business value where usability is concerned?

              > Do you have any advice on working out the maths?

              Customer determines stories based on (subjective) value. If they
              want a usability story, they'll schedule it. If they don't, they
              won't. Does that seem OK?

              Ron Jeffries
              www.XProgramming.com
              Adapt, improvise, overcome.
              --Gunnery Sergeant Tom Highway (Heartbreak Ridge)
            • leina elgohari
              I really wanted to keep the maths very very simple (ROI can make the maths complicated). I was toying around with Mynott C et al (1994) table showing the cost
              Message 6 of 13 , May 9 1:26 AM
              • 0 Attachment

                I really wanted to keep the maths very very simple (ROI can make the maths complicated). I was toying around with  Mynott C et al (1994) table showing the cost of making changes:

                Concept – 1

                Detail design – 10

                Tooling – 100

                Testing – 1000

                Post-Release – 10000

                I thought to use the above, invert the figures in the table so that at:

                Sprint/Release 1 – use 10000 to calculate the ‘simple maths’(this will give a high number thus making it imperative to carry out the usability user story(s) in this release)

                Sprint/Release 2 – use 1000 to calculate the ‘simple maths’

                Sprint/Release 3 – use 100  to caculate the 'simple maths'

                and so on…


                However, there is a problem:

                (1) My calculations assume that usability involvement peters out towards the tail-end of the project. Is that case with usability in an agile project?

                (2) Mynott's table refers to cost of making changes where the project has adopted a Waterfall approach. Is Mynott's table relevent in agile projects?



                --- On Fri, 5/9/08, William Pietri <william@...> wrote:
                From: William Pietri <william@...>
                Subject: Re: [agile-usability] Agile UCD Velocity Points
                To: agile-usability@yahoogroups.com
                Date: Friday, May 9, 2008, 2:14 AM


                Leina wrote:

                What if the usability people wanted to dedicate an entire user story to doing usability stuff.

                From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

                Do you have any advice on working out the maths?


                Well, Leina, most of the teams I coach judge value intuitively or with proximate metrics, so take this with a grain of salt.

                If I had to justify usability in that framework, I'd start by finding statistics on the kinds of improvement you get out of usability improvements. Next I'd try to get some statistics on the actual use of my app. Then I'd try to build a mathematical model of a particular flow and calculate value for usability improvements.

                For example, suppose you're at Amazon, and you suspect people are getting confused during the checkout process and are abandoning their carts.

                Step one would be to find out what sort of improvements you might see in improved usability. Let's say you decide that particular fixes to increase clarity could keep people from giving up 5% of the time.

                Then you'd look at usage statistics. Suppose that the number of abandoned carts is 1000/day, with $100 of merch in each cart, for a total of $100k left in carts.

                Next you'd do look at the steps in the flow and see what you could do. You could apply these techniques at two different stages, giving you a total of 10% reduction in lost carts, saving $10k a day, or $3.65 million annually.


                For these kinds of discussions, you hopefully can lean a lot on your client for this. They already should have a clear model of their business. It's worth just asking them how much increased customer satisfaction or reduced search time or whatever would be worth to them.

                To read more, I'd start with Jakob Nielsen's stuff on this:

                http://www.useit. com/alertbox/ roi.html
                http://www.useit. com/alertbox/ roi-first- study.html
                http://www.useit. com/alertbox/ intranet- usability. html

                And I'm sure others can suggest more material along these lines.

                My main tip, though, is to start learning to put things in business terms. Quality and enjoyment might seem a little fuzzy to some suit-wearers, but when you talk about increased customer retention, improved referrals, more purchases, or less wasted staff time, they'll pay much better attention.

                Best,

                William


                leina elgohari wrote:

                The Client would supply the figures. So an example (developer) user story reads like so:

                 

                As a trainer

                I want to present training courses

                So that Company Z can improve resource utilisation by 10% (=£8K)

                If size = 8

                Then difficulty rating =  business value / size = 100

                 

                 

                The difficulty arises when you decide to write the user story from a usability angle:

                 

                As a trainer

                I want to present quality training courses (quality here means providing an enjoyable user experience)

                How do you measure the benefits to Company Z. In other words how do you put a value on enjoyment/user satisfaction?

                 

                Regards

                Leina

                --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

                From: William Pietri <william@scissor. com>
                Subject: Re: [agile-usability] Agile UCD Velocity Points
                To: agile-usability@ yahoogroups. com
                Date: Thursday, May 8, 2008, 11:08 PM

                Interesting question, Leina!

                How does your organization measure business value for the other stories in the queue?

                William

                leina elgohari wrote:

                Thanks William

                That's a very valid point.

                What if the usability people wanted to dedicate an entire user story to doing usability stuff.

                From where would they get the business value given that its not a straightforward process to be able to quantify the business value where usability is concerned?

                Do you have any advice on working out the maths?

                 

                Regards

                Leina
                --- On Thu, 5/8/08, William Pietri <william@scissor. com> wrote:

                From: William Pietri <william@scissor. com>
                Subject: Re: [agile-usability] Agile UCD Velocity Points
                To: agile-usability@ yahoogroups. com
                Date: Thursday, May 8, 2008, 4:41 PM

                Leina wrote:
                > If the usability people are going to build in usability acceptance
                > criteria into the user stories it will no doubt add extra work.
                >
                > Do they need to estimate that work, work out their own velocity and
                > build it into the user story?
                >

                One of the things I see teams struggle with during agile adoption is
                going from a team with formal roles to a more collaborative team.

                In the beginning, when people are role oriented, database work is done
                by the database guys, back-end development is done by the server-side
                specialists, front-end development is done by the web people, and
                usability is done by the human factors expert. I think this is a natural
                place to start, but should be looked at as something to move away from
                over time.

                I think each estimated story should have a single estimate that
                represents the whole team's estimate, including everything it takes to
                get to 100% done. However, when scheduling stories within iterations,
                the team should be sensitive to individual overload and burnout.

                During iterations, teams should be working together in ways that provide
                some cross-training. If not enough of that is happening, explicitly
                schedule it in the course of working on stories. E.g., get together with
                the front-end people, and work out the UI together. It will take a
                while, but eventually your team will pick up some usability skills,
                allowing the basic stuff to be done by a broad range of people. Then the
                team can adjust organically to stories with different workloads, making
                the concept of a team velocity much more real.

                William



              • meszaros_susan
                Looks pretty complicated to me...I don t understand the terminology (without seriously considering and reading it), so why would the customer understand? I
                Message 7 of 13 , May 9 2:57 AM
                • 0 Attachment
                  Looks pretty complicated to me...I don't understand the terminology
                  (without seriously considering and reading it), so why would the
                  customer understand?

                  I think you have to speak their language, and as William pointed out a
                  simple, quick and dirty calculation on the improvement in revenue due
                  to a small change as a result of usability improvements would
                  potentially scream "fix me fix me" to the cheque writers. Make it
                  meaningful to your target and they will not only understand but act.
                  Make them work to understand and you'll lose them (gee, isn't this
                  what happens with a bad interface or poor usability?)

                  susan
                • William Pietri
                  ... Yes. This is why most teams I work with judge the value of features intuitively or through proximate measures, like user activity. The only people I have
                  Message 8 of 13 , May 9 9:58 AM
                  • 0 Attachment

                    leina elgohari wrote:
                    I really wanted to keep the maths very very simple (ROI can make the maths complicated)

                    Yes. This is why most teams I work with judge the value of features intuitively or through proximate measures, like user activity.

                    The only people I have seen successfully use rigorous money-based metrics for business value are large companies with relatively stable business models and one or more people devoted full time to tracking metrics and modeling money flow. And there I think I've only seen it used at the project level, not the story level.

                    I imagine you could also use money-based metrics in a casual way, as a way for people try to quantify their intuitions a little. I think that would only work if your business model is pretty stable, though.

                    (1) My calculations assume that usability involvement peters out towards the tail-end of the project. Is that case with usability in an agile project?


                    Not in my experience. Many of the projects I work on don't end, so I don't have a ton of data for you, but I generally see usability involvement as relatively stable. I see more usability-focused or usability-generated stories later in a project, driven from real-world data.

                    For example, just this week I saw a team release a usability-improving feature to reduce people dropping out in the middle of a flow. And now they're working on a feature to give them better metrics that they will then use to drive future usability decisions.

                    (2) Mynott's table refers to cost of making changes where the project has adopted a Waterfall approach. Is Mynott's table relevent in agile projects?

                    If it is, then we have failed, and should go back to waterfall.

                    My belief is that the cost-of-change curve has been dramatically lowered, and that other cost curves (e.g., increasing cost of distant research and planning, increasing chance of external change, increasing chance of chained errors, cost of money spent with no return) can more than compensate for the rest.

                    At least in my on-the-web world, "ship it and see what happens" isn't just a viable alternative, it's seen as superior in many cases, especially when using techniques like A/B testing.  So I think you should avoid using a mathematical model that encourages a lot of work based on speculation.


                    Hoping that helps,

                    William
                  • leina elgohari
                    Hi Susan Thanks for your advice. If it looks too complicated then... well I m not going to ditch it all together but need to rethink it through. But I ll take
                    Message 9 of 13 , May 12 2:22 AM
                    • 0 Attachment

                      Hi Susan

                      Thanks for your advice.

                      If it looks too complicated then... well I'm not going to ditch it all together but need to rethink it through.

                      But I'll take on board your advice. I thought you had to come up with something precise and concise but the message I'm getting is provide a rough guesstimate of the figures.

                      Leina

                       



                      --- On Fri, 5/9/08, meszaros_susan <susan.meszaros@...> wrote:

                      From: meszaros_susan <susan.meszaros@...>
                      Subject: [agile-usability] Re: Agile UCD Velocity Points
                      To: agile-usability@yahoogroups.com
                      Date: Friday, May 9, 2008, 10:57 AM

                      Looks pretty complicated to me...I don't understand the terminology
                      (without seriously considering and reading it), so why would the
                      customer understand?

                      I think you have to speak their language, and as William pointed out a
                      simple, quick and dirty calculation on the improvement in revenue due
                      to a small change as a result of usability improvements would
                      potentially scream "fix me fix me" to the cheque writers. Make it
                      meaningful to your target and they will not only understand but act.
                      Make them work to understand and you'll lose them (gee, isn't this
                      what happens with a bad interface or poor usability?)

                      susan

                    • Ron Jeffries
                      ... The nature of the problem, estimating velocity, and tracking it, is such that precision is essentially impossible. Imagine estimating your velocity driving
                      Message 10 of 13 , May 12 6:23 AM
                      • 0 Attachment
                        Hello, leina. On Monday, May 12, 2008, at 5:22:46 AM, you wrote:

                        > Thanks for your advice.

                        > If it looks too complicated then... well I'm not going to ditch
                        > it all together but need to rethink it through.

                        > But I'll take on board your advice. I thought you had to come up
                        > with something precise and concise but the message I'm getting is
                        > provide a rough guesstimate of the figures.

                        The nature of the problem, estimating velocity, and tracking it, is
                        such that precision is essentially impossible.

                        Imagine estimating your velocity driving across country. Estimating
                        to even the nearest mile per hour over each short interval is simply
                        silly. Tracking our speed over small intervals won't work because of
                        traffic, weather, and other factors. Keeping track of our overall
                        speed across country is interesting, but precision would be entirely
                        wasted: one extra rest stop or a slow waitress at lunch will change
                        the number substantially.

                        We find that estimating stories in small proportional integers 1, 2,
                        3, and counting the points upon completion, is sufficient for most
                        purposes.

                        For a sense of the level of precision that I think is appropriate
                        for most projects, you might enjoy this article:

                        Big Visible Charts

                        It's time to revisit the topic of Big Visible Charts. Display
                        important project information not in some formal way, not on the
                        web, not in PowerPoint, but in charts on the wall that no one can
                        miss. [Updated: Velocity Charts]

                        http://www.xprogramming.com/xpmag/BigVisibleCharts.htm

                        Regards,

                        Ron Jeffries
                        www.XProgramming.com
                        There's a difference between righteous anger and just being crabby.
                        --Barbara Richmond
                      Your message has been successfully submitted and would be delivered to recipients shortly.