4119Re: [agile-usability] Re: Online Usability Tests
- Mar 25, 2008Hi, Todd. I think we agree entirely that people should do both
quantitative and qualitative research, and both have their strengths. A
few responses to minor points.
Todd Zaki Warfel wrote:
> Well, 4 isn't enough. [...]I work mainly with startups and small companies. Perhaps you can do a
> 1. I would suspect that the "minor user interface issues" would have
> been easily corrected with simply having a good informed interaction
> designer or usability specialist assess the interface.
> 2. Did you do a 12 user study on this interface?
12-user study more effectively than they can, but it's well beyond
something a lot of small shops can afford to do on a regular basis.
Doing 4-6 people once a month is more their speed.
It may be that their designers don't meet the level you consider good.
However, they are generally the best the company has been able to find.
Whatever practices I recommend have to work in that context, which
frequently includes people wearing multiple hats.
> I'll bet that if you did, you would have found the same issues—I'veI'm sure you win that bet a lot, but in this case you would have lost.
> done this literally hundreds of times.
One substantial cause of failure was international addresses. The cost
of a multi-continent usability study surely makes sense for some people,
but not for the sums involved in this case. At the cost of a couple of
days time around the office, though, it paid off nicely.
> If you didn't, how would you know that it's beyond what you can findWell, I said that because of a little math. Perhaps I'm doing it wrong,
> from a 12 person study?
but if only 1-2% of people have some issue, the odds of finding that
particular issue in a 12-person test don't seem particularly high. And
if only 1 of 12 has a problem, it would be hard to say whether it's a
pattern or a fluke. Whereas with 10,000 data points, you'll be able to
do solid ROI calculations so that you know which fixes are worth the effort.
> We use web metrics to help identify key abandonment areas, thenI find it weird to keep saying this, but I really like in-person
> in-person field studies to find out the why. For example [...]
studies. I think they are the bees knees. Honest. I do them every chance
The only reason I got involved in this thread was to mention how on-line
testing indeed can capture some things you said it couldn't, and to
mention how some people I know are doing it. Nobody should feel obliged
to do it that way, and they certainly shouldn't stop doing in-person
- << Previous post in topic Next post in topic >>