Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] evaluating ease of use

Expand Messages
  • Daniel Naumann
    Hi Andrea, Since these are existing products you should have a wealth of info, such as, log files, completion rates and times, drop out rates and drop out
    Message 1 of 6 , Feb 3, 2009
    • 0 Attachment
      Hi Andrea,

      Since these are existing products you should have a wealth of info, such as, log files, completion rates and times, drop out rates and drop out points, etc.  And best of all you'll have existing users you can hopefully go talk to to get feedback.

      Coming up with a list of key tasks is also a good idea.  You can then evaluate these tasks using whatever criteria your company and your users think are most important.  if you need a starting point I've found Nielsen's Ten Usability Heuristics (http://www.useit.com/papers/heuristic/heuristic_list.html) a good starting point.  You'll most likely need to modify them, but it should get you going.

      I'm sure others will have more suggestions too.

      Cheers,
      Dan.

      2009/2/4 roosriders <roosriders@...>

      I am currently involved in a project to evaluate two similar products
      to determine which one should be end-of-lifed and which we should
      continue to sell. The products are web hosting platforms that
      customers access and managed via a gui interface.
      Of course this will be an extremely politically charged issue, thus I
      would like to use some recognized usability criteria for evaluation and
      ultimately move forward with the product that is easiest for our
      customers to use.
      I was thinking of creating some task based criteria. For example, how
      many steps does it take a user to create a simple website or add email
      user accounts. What other ways could i evaluate the products? Has
      anyone ever done this type of evaluation before?
      I don't want to get bogged down in how the information should be
      presented (because I out of my league on that type of evaluation) and
      will recommend that an expert take on that job.
      I look forward to any advice and guidance.
      Thank you,
      Andrea


    • marjoriepries
      Andrea, I concur with Dan. Your production support collateral should already provide a lot of clues about what your users are looking for and where they are
      Message 2 of 6 , Feb 3, 2009
      • 0 Attachment
        Andrea,

        I concur with Dan. Your production support collateral should already
        provide a lot of clues about what your users are looking for and
        where they are having problems. But I am less supportive of task
        lists as they often reflect management's bias as to what they think
        is a problem or a solution and may not be in touch with users at all.
        I would recommend you survey all the hard-evidence you already have,
        first. Then do some open-ended interviews and unstructured live
        observations with a representative sampling of your customers. What
        do they think is important? What products do they use as a basis for
        comparison? What features do they avoid using and where are they
        introducing workarounds or using external help? After you've gotten
        that broad view of the terrain, you can define more structured
        testing with defined tasks if needed to complete your product
        comparison.

        And forget to give equal attention to the plusses and well as the
        minuses. You don't want to come up with a study skewed to failure
        points at the expense of product strengths. You don't want to
        inadvertantly discard a feature or behavior that users really like
        while you're correcting a pain point.

        --- In agile-usability@yahoogroups.com, Daniel Naumann
        <danielnaumann@...> wrote:
        >
        > Hi Andrea,
        >
        > Since these are existing products you should have a wealth of info,
        such as,
        > log files, completion rates and times, drop out rates and drop out
        points,
        > etc. And best of all you'll have existing users you can hopefully
        go talk
        > to to get feedback.
        >
        .....
        > I'm sure others will have more suggestions too.
        >
        > Cheers,
        > Dan.
        >
        > 2009/2/4 roosriders <roosriders@...>
        >
        > > I am currently involved in a project to evaluate two similar
        products
        > > to determine which one should be end-of-lifed and which we should
        > > continue to sell. .......
        > > Thank you,
        > > Andrea
        > >
        > >
        > >
        >
      • Scharn, David
        Hi Andrea, You might want to review Quesenbery s article, What Does Usability Mean: Looking Beyond Ease of Use.
        Message 3 of 6 , Feb 4, 2009
        • 0 Attachment

          Hi Andrea,

           

          You might want to review Quesenbery’s article, What Does Usability Mean: Looking Beyond Ease of Use.

          http://www.wqusability.com/articles/more-than-ease-of-use.html

           

          She proposes ways to evaluate/define ease of use. Some of her characteristics are measurable (effective, efficient, error tolerant, easy to learn). These are characteristics of the product that you can quantify, which should help make your evaluation more objective. The numbers will justify your recommendation.

           

          Dave Scharn

          Interaction Designer

          Kronos

          978.947.4518

           

          From: agile-usability@yahoogroups.com [mailto:agile-usability@yahoogroups.com] On Behalf Of roosriders
          Sent: Tuesday, February 03, 2009 11:06 AM
          To: agile-usability@yahoogroups.com
          Subject: [agile-usability] evaluating ease of use

           

          I am currently involved in a project to evaluate two similar products
          to determine which one should be end-of-lifed and which we should
          continue to sell. The products are web hosting platforms that
          customers access and managed via a gui interface.
          Of course this will be an extremely politically charged issue, thus I
          would like to use some recognized usability criteria for evaluation and
          ultimately move forward with the product that is easiest for our
          customers to use.
          I was thinking of creating some task based criteria. For example, how
          many steps does it take a user to create a simple website or add email
          user accounts. What other ways could i evaluate the products? Has
          anyone ever done this type of evaluation before?
          I don't want to get bogged down in how the information should be
          presented (because I out of my league on that type of evaluation) and
          will recommend that an expert take on that job.
          I look forward to any advice and guidance.
          Thank you,
          Andrea

        • Andy Edmonds
          There is a now ISO standard method for summative , aka late stage evaluative, usability testing. The standard originated from an effort by NIST called the
          Message 4 of 6 , Feb 4, 2009
          • 0 Attachment
            There is a now ISO standard method for "summative", aka late stage
            evaluative, usability testing.

            The standard originated from an effort by NIST called the common
            industry format (CIF).
            http://www.google.com/search?q=common+industry+format
            http://zing.ncsl.nist.gov/iusr/

            This provides a highly rigorous way of designing, conducting, and even
            presenting the results of a test.

            Doing not just one but two tests may be beyond your available scope,
            in which case, I second the suggestion of using Nielsen's heuristics
            as objective criteria starting points.

            Cheers,
            Andy
            http://uxagile.com

            --- In agile-usability@yahoogroups.com, "roosriders" <roosriders@...>
            wrote:
            >
            > I am currently involved in a project to evaluate two similar
            products
            > to determine which one should be end-of-lifed and which we should
            > continue to sell. The products are web hosting platforms that
            > customers access and managed via a gui interface.
            ...
            > What other ways could i evaluate the products? Has
            > anyone ever done this type of evaluation before?
            > I don't want to get bogged down in how the information should be
            > presented (because I out of my league on that type of evaluation)
            and
          • Huey.DLL@gte.net
            Hello Andrea, I d like to comment on this issue because we re going through a similar set of events. My employer has bought up several competing products and
            Message 5 of 6 , Feb 5, 2009
            • 0 Attachment
              Hello Andrea,
               
              I'd like to comment on this issue because we're going through a similar
              set of events.  My employer has bought up several competing products
              and has sent a new manager around looking for products that can be
              end of lifed to save money.
               
              The products share nothing in common other than some of the tasks
              they perform.  Some similarities include the ability to initiate pages,
              send emails, and manage alarm systems.  All the product lines have
              an existing customer base, each of which may be offended by end-
              of-lifing a product that they are used to.
               
              I brought this issue up with management because they didn't realize
              that they were doing useless work.  You see each of the product lines
              that have overlapping functionality are stable code bases.  There are
              enhancements elsewhere but no real ongoing maintenance.
              Combining the products by eliminating "paging" from one product
              in order to use the "paging" system of another was actually more
              work at this point.
               
              Worse management bought up these competing lines, allowed
              development of all products to continue for a year and suddenly
              want to axe something.  They didn't notice that while the various
              products did compete against one another, there were clear
              preferences given by customers as to which products they
              wanted to have.  The actual overlap didn't amount to much.
              The customers have also rejected the notion that one product
              meets all potential customer needs.  Each product was created
              with a specific usability and functionality to address.  In the end,
              management has backed off the focus of actally axing products
              just to save costs.
               
              We've entered a period where each of the groups have tried to
              cooperate more and management has started to see where each
              development group has something to offer the company as a
              whole.  Some groups are better at Agile Development, one
              is better at producing features at a low cost, and another has
              some moderate success at providing a better user experience.
              At this point managment has finally taken the approach that
              they want to look at what works and then make decisions.
              Blindly cutting off projects doesn't make sense in this case.
               
              David


              Feb 3, 2009 03:09:13 PM, agile-usability@yahoogroups.com wrote:
              I am currently involved in a project to evaluate two similar products
              to determine which one should be end-of-lifed and which we should
              continue to sell. The products are web hosting platforms that
              customers access and managed via a gui interface.
              Of course this will be an extremely politically charged issue, thus I
              would like to use some recognized usability criteria for evaluation and
              ultimately move forward with the product that is easiest for our
              customers to use.
              I was thinking of creating some task based criteria. For example, how
              many steps does it take a user to create a simple website or add email
              user accounts. What other ways could i evaluate the products? Has
              anyone ever done this type of evaluation before?
              I don't want to get bogged down in how the information should be
              presented (because I out of my league on that type of evaluation) and
              will recommend that an expert take on that job.
              I look forward to any advice and guidance.
              Thank you,
              Andrea
            Your message has been successfully submitted and would be delivered to recipients shortly.