Re: [agile-usability] A simple risk-return-model for prioritizing user stories
Thank you for that as I think I need to add more on the users perceived time on task.
It depends on your definition of design. If the design is there to enhance the content, then yes this would help in evaluating design A vs design B. If we use the users perceived time on task.
"Time runs by when you are having fun. "
The later versions of Generalised Cost use a higher user cost per minute when the user is stuck in traffic, or waiting for a bus.
I remember reading somewhere that people overestimated the amount of time it took then to do the dishes, while under estimated the amount of time that it took for them to play a video game. It is also the reason casinos never have clocks.
I know Microsoft Reserch did some experiments on user perceived time (and I will try digging it up).....
There are several questions to this approach... 1) what is granularity of "perceived time" 2) how do we measure it 3).......
Actual time on task can give us an idea of where the problems lie. If a user is spending too much time filling in a form field then the user may be finding the question challenging.
I wondering if your answer on both Satisfying and mean-end analysis could help answer Robert's question as well. With Simon's theory of "Satisfying" are we trying to Satisfice as fast as possible?
If that is the case is there not a challenge if I am doing something fun like playing a computer game, I am wanting to complete the game fast, but my enjoyment is higher if the game delays challenges me.
Thanks for that feedback....
So you'll get more uptake if you write more in the style of a cookbook or game instructions than a treatise.
I will follow your advice here and next version will be more of cookbook.
Because so many usability factors are subliminal, anything that raises their value to a conscious level could help project stakeholders properly value usability.
My driving force here was to try to get all the inputs valued. Agile in my experience is very strong when there is one customer driving the priority of each story. When you start adding stake holders then the picture gets more hazy. I hope this method can clear this up.
I am also thinking this method could be used to justify Usability improvements to current system.
But it needs stress testing and hence the email to the list.
All the best
JamesOn Mon, Jun 30, 2008 at 11:32 PM, Robert Biddle <Robert_Biddle@...> wrote:
How about explaining how this relates to design?
In particular, might two people with access to the same information
may come up with different designs, yet one be vastly
better than the other?
Does the model address this?
James Page wrote:
> On this mailing list there has been much conversation about how to
> work out the budget in time and cost of an Agile project. The other
> question that appears is where the design of a user interface should
> sit in the development cycle.
> I think by bringing in a simple risk-return-model for prioritizing the
> user stories could help to answer these questions.
> I am writing this piece to get as many comments back to this approach,
> so that it can be modified and improved upon.
> When looking at Agile Development it is interesting to look at the
> Nobel Prize winner Herb Simmons work. Herb Simon's theory *bounded
> rationality* says that perfectly rational decisions are often not
> feasible in practice due to the finite computational resources
> available for making them. His models assume that the human forms a
> Goal and then an intermediate sub goal. When you reach the sub goal
> you then form a new sub goal on the journey to reach the end goal. In
> other words you reach your goal by making many sub-goals. But you only
> plan the immediate sub-task.
> If you look at Agile we define what the project is for and then
> develop stories. At the start of the project we do not define the
> order that stories are done in. During each iteration we define which
> stories take place within the iteration. So we can say that sub goals
> are the equivalent of an iteration or sprint.
> In contrast the Waterfall approach follows a more rationalised
> economic approach where all the sub goals are defined for reaching the
> end goal. This is where we break up the project into milestones were
> we define the sub goals at the start of the project.
> The challenge is that the Agile methods only loosely define how we
> identify the sub goals, except to say that they should be the most
> important or critical for the business.
> Looking at an Agile project in the terms of *bounded rationality
> *allows us to use some of the economic methods that have been defined
> from the field. For example in estimating the time-frame, or budget of
> a project we can use top down forecasting methods, instead of the
> normal bottom up approach.
> It also helps sorting out the confusion with Usability and Agile in
> answering the question is Usability part of the sub-goal, or is it
> defining it. We can divide the answer to Usability into three areas.
> 1) Anything that defines a goal belongs outside the iteration and is a
> story. 2) Anything that is about meeting the goal or the requirements
> of the iteration belongs within it, and 3) any Usability requirements
> are the stories acceptance test.
> Using micro economics and converting everything to a monetary value
> also allows us then compare the different inputs and outputs of a
> project. We lose the problem of comparing apples and oranges. People
> need to be aware that the Cost and the Benefit will change during the
> project, and that the numbers are an estimate.
> The method allows us to drive the project forward by creating one
> customer (in an Agile sense) by combing both the business requirements
> and the users. This is because we have converted Usability into a
> monetary metric.
> We can prioritise the stories using a simple economic model looking at
> the Risk/Return for each story. The model would assume that we want to
> minimise as much risk as possible early in the project, as well as
> have the largest return. We can now go back to the agile methods which
> would say that if the User Interface has both a high Return and a high
> risk then it should be done as early as possible. If it carries little
> risk and little return then it can be done later on.
> How do we define the two inputs? Risk can be calculated by slightly
> modifying the technique of the team estimating the amount of work for
> each story. The greater the divergence between the team members
> estimates equates to a greater risk. Another technique could be
> getting each team member to estimate a maximum and minimum amount of
> work for the story and again looking at the range for each story.
> The work value is easy to convert back into monitory value as we
> convert the work value anyway into time to calculate the burn down
> chart. Once we have a time value we just divide the cost of the team
> by time.
> The return number could be as simple an estimate of the economic
> benefit of the story to the firm.
> Another method suggested to me by the transport Economist Chris Cheek
> is using the "Generalised Cost" model. This would be more user
> friendly. The "Generalised Cost" model is from Transport economics and
> is used by transport planners to prioritise where investment into rail
> or road should take place. Generalised Cost model uses the time value
> of a user and works out the cost to them using a mode of transport
> taking into account the value of the users' time.
> There are many different "Generalised Cost" models in use, but in the
> whole they use a "cost" for each segment of the population, normally
> dependent on the users' activity. For example a Business user would
> have a higher cost per hour than leisure users. Then the actual cost
> of the mode of the transport is added to the user cost. Some more
> complex models also try to account for user's perceived time. For
> example they use a higher cost per hour when the user is stuck in
> traffic, or they use a lower cost per hour for train travel as the
> user can carry out other activities while they travel.
> We can apply "Generalised Cost" to Agile Usability by calculating the
> time that the user would spend per task. We can use a higher Cost for
> users that are under a higher cognitive load, for example when the
> user makes an error.
> We can calculate the value created for the user by comparing the new
> design with an old design or competitors design. At the early stage of
> the project we can use one of the GOMS methods for working out an
> estimate of the time that a user would take to complete the task, and
> later on we can use real user times from lab tests.
> So by using the "Generalised Cost" model we can work out the value the
> user captures by using the new Interface rather then the old interface.
> Tim O'Reilly (of the publishers) says that "Create more value than you
> capture." In other words a good internet business should always create
> more value for its users then the business creates. We could even use
> this as a metric by comparing the economic benefit to the firm versus
> the saving the user makes by using the new interface.
> To answer the question that I posed at the start of this where does
> the Usability then fit into the Agile project we can look at the
> different inputs that we need for the model. Any tasks that the
> project requires are a "story" and therefore should be defined outside
> the iteration.
> The interface design does not change the story, but its implementation
> does effect the "critical parameters" therefore it belongs inside the
> iteration. For example "a user visits the site and purchases X" would
> be a story.
> The critical parameters of each of the stories will drive the
> acceptance test. These parameters are the inputs to the user's
> Generalized Cost, these are the usual usability evaluation methods.
> In conclusion it does not matter if you use a Generalized Cost method
> or not but by looking at the model we can see where Usability could
> fit into the Agile method, and which stories need should have a higher
> I would be grateful for any feedback, or corrections.
> All the best