RE: [agile-usability] Best methods for user feedback before release
Its like the example I read recently on another list about the question asked to a fast-food chain’s customers, ‘Would you buy a healthier version of our burgers?’. In the example, many people said ‘yes’ but this didn’t result in increased sales when one was produced because although people felt they should say yes, in reality they wanted the good old high fat version. These kind of factors can of course be compounded in group situations.
I can see some benefit of group situations though, in that discussion might encourage users to think about, and explore, issues beyond what they thought were important (OK, a researcher can encourage this in a one–to-one, but it is only after talking to a few people will they have a wide enough range of feedback to enable them to steer the user to different areas that the user might not have considered so far - but have been raised by other users).
Senior Web Producer
01865 476596 itillier@...
Please do not print this e-mail
On Jul 7, 2008, at 3:33 PM, Nick Gassman wrote:
but to get a reaction from a group about the type of features they want and to stimulate some discussion I think it's quite useful. Are you saying that focus groups are no good for any type of research, or just some?
You can get this same feedback on-on-one during a usability session. In fact, during a usability session, you're feedback is even richer, as it has context and doesn't suffer from group think. Not to mention that what people say they want and what they actually use are often quite different. You're better off doing some ethno! graphic based field research and watching for things people need, than asking a bunch of people "what they think they want."
They can be useful as a validation technique, but there are so many other methods, I don't personally put a lot of value in them.
Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
Contact Info Voice: (215) 825-7423 Email: todd@... AIM: twarf! el@... Blog: http://toddwarfel.com Twitter: zakiwarfel ---------------------------------- In theory, theory and practice are the same. In practice, they are not.
This email (and any attachment) is confidential, may be legally privileged and is intended solely for the
use of the individual or entity to whom it is addressed. If you are not the intended recipient please do
not disclose, copy or take any action in reliance on it. If you have received this message in error please
tell us by reply and delete all copies on your system.
Although this email has been scanned for viruses you should rely on your own virus check as the sender
accepts no liability for any damage arising out of any bug or virus infection. Please note that email
traffic data may be monitored and that emails may be viewed for security reasons.
Blackwell Publishing Limited is a private limited company registered in England with registered number 180277.
Registered office address: The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ.
- Larry Constantine wrote:
Years of horror stories from clients have convinced me that even for investigating concepts and what features users/customers want, focus groups are dangerously misleading. First, people do not know what they want; second, what they say is often loosely related if at all with what they want; third, what they want is only weakly correlated with what they need; and fourth, group and social effects dominate over directness in focus groups, even when led by relatively good facilitators. In the end, you really have no idea what the relationship between stated or summarized “findings” and genuine needs and interests. I have literally seen companies invest millions in product and Web development based on and validated by focus groups, only to discover that they had built the wrong system the wrong way.
Although I am also very suspicious of focus groups, I have seen people get value out of something weirdly similar: parties and get-togethers.
Here in the land of startups, it's pretty common for people to have launch parties, in-office happy hours, and casual get-togethers with staff, users, and other stakeholders. There is naturally a lot of conversation around topics similar to what one might try to use a focus group to address. People seem to get a lot of value out of that conversation, walking away with ideas, background information, and a better appreciation for the lives of users.
Perhaps the key difference is in the informality. Nobody would think of writing up "findings" for a party; people know to treat the information gained very lightly.