Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Re: Online Usability Tests

Expand Messages
  • William Pietri
    ... Ah, I see. Sorry for the confusion. On one occasion, where credit card usage was the primary focus, the system had already been designed to record every
    Message 1 of 29 , Mar 25, 2008
    • 0 Attachment
      Todd Zaki Warfel wrote:
      > Sorry, should have been more specific. What I'm interested in is
      > exactly how you "instrumented things," or what you did to capture
      > everything so you were able to tell exactly what individual
      > fields/items were the culprit. I'd love to know more about this
      > technique as an option to use in the future.

      Ah, I see. Sorry for the confusion.

      On one occasion, where credit card usage was the primary focus, the
      system had already been designed to record every request sent to the
      credit card processor, plus processor responses, so that was a rich seam
      of data to mine. After various UI changes, we then monitored to make
      sure that we indeed solved the problems.

      On another, we found the particular place in the code to which where a
      form was submitted, and logged as much information as possible,
      including IP, browser info, and the details of the form submission. Then
      a little perl cleaned the output enough to feed it to a business
      analyst, who worked in collaboration with the designer to figure out
      what particular failures meant and how to fix them.

      On a third project, we started out wondering these things early, and so
      had one bit of code near the heart of things that logged every bit of
      user input. I don't remember any formal studies that used it, but we'd
      often use it to answer some particular question, or to see what users
      were up to. That was especially handy when discussing whether or not
      users would really do something.

      The imagined ajaxification of this would be a little more complicated,
      collecting client-side events (like keypresses and mouse movements) and
      state information (transaction ids, current state of forms) and
      uploading them via background asynchronous requests. It'd be exciting to
      dig through that data, but one of the first things I'd look at is failed
      client-side validation. Another would be the amount of time and rework
      for individual fields.

      People who are doing advanced work in this area include Netflix and
      Google. They had a great BayCHI presentation which is, alas, not up on
      the web yet, but I will mention it here when it is.

      Hoping that helps,

      William
    Your message has been successfully submitted and would be delivered to recipients shortly.