Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Usability Test Goals

Expand Messages
  • Ash Donaldson
    ... Blatantly biasing the participant from the beginning. ... The goal of a usability test is to evaluate a proposed hypothesis. This is why it is recommended
    Message 1 of 9 , Aug 30, 2005
    • 0 Attachment
      On 31/08/2005, at 5:35 AM, bunkybrian wrote:
      I just saw a usability test where the practitioners were Information
      Architects.  One of the comments in this test was "the goal of this
      usability test is to validate my new design."

      Blatantly biasing the participant from the beginning.

      From some reason, I have a problem with this statement.  It seems
      like the goal of the usability test should be to obtain customer
      feedback about a new design.

      The goal of a usability test is to evaluate a proposed hypothesis.  This is why it is recommended that an objective third party, NOT the designer test the validity of any proposed designs.

      In my mind, the usability practioner is already starting out with
      the mindset that their new design is correct.  The design just
      needed to be validated by the user.

      This 'practitioner' sounds like one of the cowboys that half-read a couple of books and just started calling themselves an Information Architect (and charging accordingly).

      In addition, there were leading questions throughout the sessions
      that directed the user towards a conclusion rather than having the
      user give unsolicitated feedback.

      No. No. No!

      As you can tell, I am somewhat disappointed in this test session.  I
      wanted to get some feedback from this group as to whether the goal
      of usability testing is to validate a new design versus obtaining
      feedback from the user or something else.

      Understandably so .   I'd be advising the Information Architect's client to get their money back and make sure that they and their colleagues never use that IA or company again.  Before contracting user-centred design service like IA or usability, clients really should look for someone with some sort of qualifications (HCI, Human Factors, Social or Cog. Psych.) or at the very least, industry recommendations.


      Best regards,


      Ash Donaldson

      OZCHI 2005 Conference Chair

      chair@...


      OZCHI 2005

      Citizens Online: Considerations for today & the future

      www.ozchi.org




    • Donna Maurer
      ... Yes, that s bad. A more usual goal would be to identify whether usability issues exist that would cause people difficulty in achieving what they need to
      Message 2 of 9 , Aug 31, 2005
      • 0 Attachment
        On 30 Aug 2005 at 19:35, bunkybrian wrote:

        > Hello --
        >
        > I just saw a usability test where the practitioners were Information
        > Architects. One of the comments in this test was "the goal of this
        > usability test is to validate my new design."
        >
        > >From some reason, I have a problem with this statement. It seems
        > like the goal of the usability test should be to obtain customer
        > feedback about a new design.

        Yes, that's bad. A more usual goal would be to identify whether usability issues exist
        that would cause people difficulty in achieving what they need to do. This is less likely
        to cause bias.

        But the 'information architect' role here is irrelevant. Some IAs have good usability
        testing skills, some wouldn't know a good usability test if it jumped out and bit them.
        Usability testing skills aren't necessary for IA any more than they are for business
        analysis or graphic design. Just one more skillset that some people have and some
        don't.

        Donna
        --
        Donna Maurer
        Maadmob Interaction Design

        e: donna@...
        work: http://maadmob.com.au/
        blog: http://maadmob.net/donna/blog/
        AOL IM: maadmob
      • Ron Jeffries
        ... I m with you. While we certainly hope that some new design is a good one, it seems that a more unbiased outlook could only be helpful. Ron Jeffries
        Message 3 of 9 , Aug 31, 2005
        • 0 Attachment
          On Tuesday, August 30, 2005, at 3:35:06 PM, bunkybrian wrote:

          > As you can tell, I am somewhat disappointed in this test session. I
          > wanted to get some feedback from this group as to whether the goal
          > of usability testing is to validate a new design versus obtaining
          > feedback from the user or something else.

          I'm with you. While we certainly hope that some new design is a good
          one, it seems that a more unbiased outlook could only be helpful.

          Ron Jeffries
          www.XProgramming.com
          Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
          of his head. It is, as far as he knows, the only way of coming downstairs,
          but sometimes he feels that there really is another way, if only he could
          stop bumping for a moment and think of it. And then he feels that perhaps
          there isn't. -- A. A. Milne
        • bunkybrian
          Will -- This project is an agile one. Personally, I am of the philosophy that processes do not work. It is people that do the work (which is one of the
          Message 4 of 9 , Aug 31, 2005
          • 0 Attachment
            Will --

            This project is an agile one.

            Personally, I am of the philosophy that processes do not work. It
            is people that do the work (which is one of the principles defined
            by the Agile Manifesto).

            I have since found out that a Third Party firm is conducting the
            Usability Sessions, but this firm is primarily focused in the area
            of Market Research rather than Usability.

            Some of the leading questions might have been that the Third Party
            firm is just trying to please their customer. Of course, a lack of
            usability expertise is also a factor here.

            To me, it appears that Usability is just a checkpoint for this team,
            which is truly sad. We do have a Human Factors Center that could be
            used at my firm, which I am the Usability Lead.

            Politics is preventing this team from using us.

            Anyway, thanks for the feedback. I appreciate everyone's comments.

            Bye,
            Brian

            --- In agile-usability@yahoogroups.com, William Pietri
            <william@s...> wrote:
            > On Tue, 2005-08-30 at 19:35 +0000, bunkybrian wrote:
            > > I just saw a usability test where the practitioners were
            Information
            > > Architects. One of the comments in this test was "the goal of
            this
            > > usability test is to validate my new design."
            > >
            > > From some reason, I have a problem with this statement. It
            seems
            > > like the goal of the usability test should be to obtain customer
            > > feedback about a new design.
            >
            > I'd agree.
            >
            > Every new version is a collection of gambles, and you get feedback
            to
            > see which ones are good, which could be improved, and hints on how
            to
            > improve. If you're only looking for validation, that misses much
            of the
            > point.
            >
            > Do you know what motivates this attitude for these people?
            >
            > I'd also wonder how agile this project is. To me one of the most
            > insidious things about the BDUF approach is that any honest person
            has
            > to admit that a big spec involves a lot of guesses. But people who
            fund
            > big software projects are rarely comfortable dropping hundreds of
            > thousands of dollars on guesswork; they want certainty. Especially
            in
            > competitive situations, this encourages designers to express more
            > confidence than is warranted.
            >
            > I think a natural corollary of that is seeking only validation, not
            > feedback. If somebody is busy insisting that the design is great,
            it's
            > hard to honestly look for weaknesses.
            >
            > William
            >
            > --
            > William Pietri <william@s...>
          • Jon Meads
            Brian, What a lot of people don t realize is that good usability testing requires (1) Designing the test to answer specific questions about the usability of
            Message 5 of 9 , Aug 31, 2005
            • 0 Attachment
              Brian,

              What a lot of people don't realize is that good usability testing requires
              (1) Designing the test to answer specific questions about the usability of
              the product and (2) A good facilitator who understands how to conduct the
              test without biasing the test participant.

              I've seen an awful lot of "usability tests" done by untrained people that
              are truly awful - they tend to range from demos ("Here's what you can do.
              What do you think of that?") to accolade seeking affirmations such as the
              one you mention ("Do you like my baby? Ain't she beautiful?").

              Good usability testing, like good system testing, requires someone who
              understands what they are doing and how to do it right.

              Cheers,
              jon

              -----Original Message-----
              From: agile-usability@yahoogroups.com
              [mailto:agile-usability@yahoogroups.com] On Behalf Of bunkybrian
              Sent: Tuesday, August 30, 2005 12:35 PM
              To: agile-usability@yahoogroups.com
              Subject: [agile-usability] Usability Test Goals

              Hello --

              I just saw a usability test where the practitioners were Information
              Architects. One of the comments in this test was "the goal of this
              usability test is to validate my new design."

              >From some reason, I have a problem with this statement. It seems
              like the goal of the usability test should be to obtain customer feedback
              about a new design.

              In my mind, the usability practioner is already starting out with the
              mindset that their new design is correct. The design just needed to be
              validated by the user.

              In addition, there were leading questions throughout the sessions that
              directed the user towards a conclusion rather than having the user give
              unsolicitated feedback.

              As you can tell, I am somewhat disappointed in this test session. I wanted
              to get some feedback from this group as to whether the goal of usability
              testing is to validate a new design versus obtaining feedback from the user
              or something else.

              Thanks,
              Brian Sullivan







              ------------------------ Yahoo! Groups Sponsor --------------------~--> Get
              Bzzzy! (real tools to help you find a job). Welcome to the Sweet Life.
              http://us.click.yahoo.com/A77XvD/vlQLAA/TtwFAA/dpFolB/TM
              --------------------------------------------------------------------~->


              Yahoo! Groups Links
            • William Pietri
              ... Indeed! I agree completely. I m sad to hear that they ve gone to the trouble of following a process without embracing the spirit behind it. That s a good
              Message 6 of 9 , Aug 31, 2005
              • 0 Attachment
                On Wed, 2005-08-31 at 14:20 +0000, bunkybrian wrote:
                > Personally, I am of the philosophy that processes do not work.[...]
                > To me, it appears that Usability is just a checkpoint for this team,
                > which is truly sad.

                Indeed! I agree completely. I'm sad to hear that they've gone to the
                trouble of following a process without embracing the spirit behind it.
                That's a good way to do a lot of work for relatively little benefit.

                Hopefully, though, the practice of frequent releases will help. If they
                keep hearing from their users that the thing is a pain, maybe they'll do
                their usability testing more frequently and more sincerely.

                William

                --
                William Pietri <william@...>
              Your message has been successfully submitted and would be delivered to recipients shortly.