Re: [agile-usability] Re: Online Usability Tests
- Todd Zaki Warfel wrote:
> Sorry, should have been more specific. What I'm interested in isAh, I see. Sorry for the confusion.
> exactly how you "instrumented things," or what you did to capture
> everything so you were able to tell exactly what individual
> fields/items were the culprit. I'd love to know more about this
> technique as an option to use in the future.
On one occasion, where credit card usage was the primary focus, the
system had already been designed to record every request sent to the
credit card processor, plus processor responses, so that was a rich seam
of data to mine. After various UI changes, we then monitored to make
sure that we indeed solved the problems.
On another, we found the particular place in the code to which where a
form was submitted, and logged as much information as possible,
including IP, browser info, and the details of the form submission. Then
a little perl cleaned the output enough to feed it to a business
analyst, who worked in collaboration with the designer to figure out
what particular failures meant and how to fix them.
On a third project, we started out wondering these things early, and so
had one bit of code near the heart of things that logged every bit of
user input. I don't remember any formal studies that used it, but we'd
often use it to answer some particular question, or to see what users
were up to. That was especially handy when discussing whether or not
users would really do something.
The imagined ajaxification of this would be a little more complicated,
collecting client-side events (like keypresses and mouse movements) and
state information (transaction ids, current state of forms) and
uploading them via background asynchronous requests. It'd be exciting to
dig through that data, but one of the first things I'd look at is failed
client-side validation. Another would be the amount of time and rework
for individual fields.
People who are doing advanced work in this area include Netflix and
Google. They had a great BayCHI presentation which is, alas, not up on
the web yet, but I will mention it here when it is.
Hoping that helps,