Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] In Country Web Usability Testing

Expand Messages
  • Mitchell Gass
    ... If your goal is to measure the current usability of the services, thereby establishing a baseline against which you can measure changes in future versions,
    Message 1 of 2 , Sep 5, 2008
      At 04:00 AM 9/5/2008, Harjinder Hothi wrote:
      >Our core business is offering secure, reliable and versatile online
      >payment and banking services...
      >The bank is operation and recently launched in UK, Ireland, Spain,
      >Germany, Italy, France and Canada, soon to be launched in many other
      >European countries...Ivobank has tested systems for functionality,
      >load and performance but now need to test the actual experience from
      >within the native country for the real end user experience. How can
      >this be best achieved?

      If your goal is to measure the current usability of the services,
      thereby establishing a baseline against which you can measure changes
      in future versions, you need some form of quantitative testing.


      If your goal is simply to improve the services, iterative diagnostic
      usability testing, in which you

      - have representative end users perform tasks that your services are
      designed to support

      - observe obstacles to success

      - make changes to remove the obstacles

      is faster and much more cost effective


      The challenge for you is that you'd be doing the testing only after
      completing the system, when making changes is likely to be the most
      expensive it's been in the entire development cycle. Had you tested
      early prototypes that are quick to build and that you can comfortably
      change without throwing out the results of considerable development
      effort, it's likely you could have made significant improvements to
      the user interface for a tiny fraction of what it will cost you now.
      When choosing the medium for early prototypes, what's most important
      is being able to make changes quickly. Paper prototypes can be
      excellent for this, and Carolyn Snyder's book


      does a great job of explaining how to conduct usability tests with
      early prototypes, and I expect that Todd Warfel's forthcoming book


      will also be good. Todd recently did a workshop on Agile and Paper
      Prototyping at Agile 2008


      Regardless of the testing approach you use, you'll need to decide how
      many participants to include and from which countries. If you can
      afford to be patient, the most cost-effective approach would be to
      start with just a couple of the countries you serve, find and address
      problems that are likely to be common to all the countries, and only
      then do broader testing to uncover problems that are specific to each
      country. It's best to conduct test sessions in the native language of
      each country, and you'll need to weigh the advantages of

      - having fewer test facilitators who work with interpreters, which
      lets them see firsthand the differences in different countries

      - having facilitators who are fluent in the language of each country
      in which you'll test.

      Finally, remote usability testing, which uses an online conferencing
      service like WebEx or GoToMeeting so that participants and observers
      do not need to be at the same location, can greatly reduce the cost
      of the testing. It's also more comfortable for participants, who can
      join from home or work rather than have to travel to a research
      facility. These days, most of the usability tests I conduct are
      remote, and with adjustments to the way I facilitate, I estimate that
      I'm getting at least 80% of the data from remote tests that I get
      from in-person tests. The most important adjustments are

      - Ask more questions

      - Encourage participants to think aloud even more than in in-person sessions

      I can then use verbal cues in place of visual cues to understand what
      participants are experiencing. The main reason that we don't all the
      data we get from in-person tests is that it takes more time to
      establish the necessary rapport with participants and to ask more
      questions. I've found that the types and quality of the observations
      are very similar to those we get from in-person tests; only the
      quantity is different.


      Mitchell Gass
      uLab | PDA: Learning from Users | Designing with Users
      Berkeley, CA 94707 USA
      +1 510 525-6864 office
      +1 415 637-6552 mobile
      +1 510 525-4246 fax
    Your message has been successfully submitted and would be delivered to recipients shortly.