Loading ...
Sorry, an error occurred while loading the content.

Few Users in Evaluation Session and Avoiding Leading Questions

Expand Messages
  • Leina
    I have to conduct usability evaluations with real users. I initially planned to have 4 users against each user group but unfortunately this has been negotiated
    Message 1 of 2 , Jun 22, 2008
    • 0 Attachment
      I have to conduct usability evaluations with real users. I initially
      planned to have 4 users against each user group but unfortunately this
      has been negotiated down to 1 user for each user group. I have a lot of
      tasks that I want that one user to complete. If I had 4 users then I
      would expect them to mention some of the areas of contention* I have
      listed prior to evaluation. Since I now have one user then I'm pinning
      all my hopes on them raising those issues I have listed. If they don't
      then I'll have to raise it with them. I just wandered if that's a sound
      approach or is that almost like asking a leading question?



      * That's me pre-empting what might come up during the evaluation or
      areas that I want users to provide further clarification on
    • Theo Mandel, Ph.D.
      You are in a tough spot. You should try to get more participants per user group or reduce the number of different user groups to get more subjects per group.
      Message 2 of 2 , Jun 23, 2008
      • 0 Attachment
        You are in a tough spot. You should try to get more participants per user group or reduce the number of different user groups to get more subjects per group. Having only one participant per user group extremely limits your ability to analyze results by user group. You will have no way of telling whether differences between groups are actually individual differences.
         
        You mentioned you already have a list of usability issues you hope to get feedback on during the evaluation. First, you should design the evaluation tasks to allow participants to experience those functional/workflow/usability issues or areas you are interested in. You must avoid asking leading questions and directing participants during the evaluation tasks. However, after completing tasks and after the session, you can certainly ask for their feedback on troublesome issues they experienced during the evaluation or even issues they may not have come across or commented on during the evaluation.
         
        Theo Mandel, Ph.D.
        Healthcare Chairman, World Usability Day 2007
         
      Your message has been successfully submitted and would be delivered to recipients shortly.