Loading ...
Sorry, an error occurred while loading the content.

evaluating ease of use

Expand Messages
  • roosriders
    I am currently involved in a project to evaluate two similar products to determine which one should be end-of-lifed and which we should continue to sell. The
    Message 1 of 6 , Feb 3, 2009
    View Source
    • 0 Attachment
      I am currently involved in a project to evaluate two similar products
      to determine which one should be end-of-lifed and which we should
      continue to sell. The products are web hosting platforms that
      customers access and managed via a gui interface.
      Of course this will be an extremely politically charged issue, thus I
      would like to use some recognized usability criteria for evaluation and
      ultimately move forward with the product that is easiest for our
      customers to use.
      I was thinking of creating some task based criteria. For example, how
      many steps does it take a user to create a simple website or add email
      user accounts. What other ways could i evaluate the products? Has
      anyone ever done this type of evaluation before?
      I don't want to get bogged down in how the information should be
      presented (because I out of my league on that type of evaluation) and
      will recommend that an expert take on that job.
      I look forward to any advice and guidance.
      Thank you,
      Andrea
    • Daniel Naumann
      Hi Andrea, Since these are existing products you should have a wealth of info, such as, log files, completion rates and times, drop out rates and drop out
      Message 2 of 6 , Feb 3, 2009
      View Source
      • 0 Attachment
        Hi Andrea,

        Since these are existing products you should have a wealth of info, such as, log files, completion rates and times, drop out rates and drop out points, etc.  And best of all you'll have existing users you can hopefully go talk to to get feedback.

        Coming up with a list of key tasks is also a good idea.  You can then evaluate these tasks using whatever criteria your company and your users think are most important.  if you need a starting point I've found Nielsen's Ten Usability Heuristics (http://www.useit.com/papers/heuristic/heuristic_list.html) a good starting point.  You'll most likely need to modify them, but it should get you going.

        I'm sure others will have more suggestions too.

        Cheers,
        Dan.

        2009/2/4 roosriders <roosriders@...>

        I am currently involved in a project to evaluate two similar products
        to determine which one should be end-of-lifed and which we should
        continue to sell. The products are web hosting platforms that
        customers access and managed via a gui interface.
        Of course this will be an extremely politically charged issue, thus I
        would like to use some recognized usability criteria for evaluation and
        ultimately move forward with the product that is easiest for our
        customers to use.
        I was thinking of creating some task based criteria. For example, how
        many steps does it take a user to create a simple website or add email
        user accounts. What other ways could i evaluate the products? Has
        anyone ever done this type of evaluation before?
        I don't want to get bogged down in how the information should be
        presented (because I out of my league on that type of evaluation) and
        will recommend that an expert take on that job.
        I look forward to any advice and guidance.
        Thank you,
        Andrea


      • marjoriepries
        Andrea, I concur with Dan. Your production support collateral should already provide a lot of clues about what your users are looking for and where they are
        Message 3 of 6 , Feb 3, 2009
        View Source
        • 0 Attachment
          Andrea,

          I concur with Dan. Your production support collateral should already
          provide a lot of clues about what your users are looking for and
          where they are having problems. But I am less supportive of task
          lists as they often reflect management's bias as to what they think
          is a problem or a solution and may not be in touch with users at all.
          I would recommend you survey all the hard-evidence you already have,
          first. Then do some open-ended interviews and unstructured live
          observations with a representative sampling of your customers. What
          do they think is important? What products do they use as a basis for
          comparison? What features do they avoid using and where are they
          introducing workarounds or using external help? After you've gotten
          that broad view of the terrain, you can define more structured
          testing with defined tasks if needed to complete your product
          comparison.

          And forget to give equal attention to the plusses and well as the
          minuses. You don't want to come up with a study skewed to failure
          points at the expense of product strengths. You don't want to
          inadvertantly discard a feature or behavior that users really like
          while you're correcting a pain point.

          --- In agile-usability@yahoogroups.com, Daniel Naumann
          <danielnaumann@...> wrote:
          >
          > Hi Andrea,
          >
          > Since these are existing products you should have a wealth of info,
          such as,
          > log files, completion rates and times, drop out rates and drop out
          points,
          > etc. And best of all you'll have existing users you can hopefully
          go talk
          > to to get feedback.
          >
          .....
          > I'm sure others will have more suggestions too.
          >
          > Cheers,
          > Dan.
          >
          > 2009/2/4 roosriders <roosriders@...>
          >
          > > I am currently involved in a project to evaluate two similar
          products
          > > to determine which one should be end-of-lifed and which we should
          > > continue to sell. .......
          > > Thank you,
          > > Andrea
          > >
          > >
          > >
          >
        • Scharn, David
          Hi Andrea, You might want to review Quesenbery s article, What Does Usability Mean: Looking Beyond Ease of Use.
          Message 4 of 6 , Feb 4, 2009
          View Source
          • 0 Attachment

            Hi Andrea,

             

            You might want to review Quesenbery’s article, What Does Usability Mean: Looking Beyond Ease of Use.

            http://www.wqusability.com/articles/more-than-ease-of-use.html

             

            She proposes ways to evaluate/define ease of use. Some of her characteristics are measurable (effective, efficient, error tolerant, easy to learn). These are characteristics of the product that you can quantify, which should help make your evaluation more objective. The numbers will justify your recommendation.

             

            Dave Scharn

            Interaction Designer

            Kronos

            978.947.4518

             

            From: agile-usability@yahoogroups.com [mailto:agile-usability@yahoogroups.com] On Behalf Of roosriders
            Sent: Tuesday, February 03, 2009 11:06 AM
            To: agile-usability@yahoogroups.com
            Subject: [agile-usability] evaluating ease of use

             

            I am currently involved in a project to evaluate two similar products
            to determine which one should be end-of-lifed and which we should
            continue to sell. The products are web hosting platforms that
            customers access and managed via a gui interface.
            Of course this will be an extremely politically charged issue, thus I
            would like to use some recognized usability criteria for evaluation and
            ultimately move forward with the product that is easiest for our
            customers to use.
            I was thinking of creating some task based criteria. For example, how
            many steps does it take a user to create a simple website or add email
            user accounts. What other ways could i evaluate the products? Has
            anyone ever done this type of evaluation before?
            I don't want to get bogged down in how the information should be
            presented (because I out of my league on that type of evaluation) and
            will recommend that an expert take on that job.
            I look forward to any advice and guidance.
            Thank you,
            Andrea

          • Andy Edmonds
            There is a now ISO standard method for summative , aka late stage evaluative, usability testing. The standard originated from an effort by NIST called the
            Message 5 of 6 , Feb 4, 2009
            View Source
            • 0 Attachment
              There is a now ISO standard method for "summative", aka late stage
              evaluative, usability testing.

              The standard originated from an effort by NIST called the common
              industry format (CIF).
              http://www.google.com/search?q=common+industry+format
              http://zing.ncsl.nist.gov/iusr/

              This provides a highly rigorous way of designing, conducting, and even
              presenting the results of a test.

              Doing not just one but two tests may be beyond your available scope,
              in which case, I second the suggestion of using Nielsen's heuristics
              as objective criteria starting points.

              Cheers,
              Andy
              http://uxagile.com

              --- In agile-usability@yahoogroups.com, "roosriders" <roosriders@...>
              wrote:
              >
              > I am currently involved in a project to evaluate two similar
              products
              > to determine which one should be end-of-lifed and which we should
              > continue to sell. The products are web hosting platforms that
              > customers access and managed via a gui interface.
              ...
              > What other ways could i evaluate the products? Has
              > anyone ever done this type of evaluation before?
              > I don't want to get bogged down in how the information should be
              > presented (because I out of my league on that type of evaluation)
              and
            • Huey.DLL@gte.net
              Hello Andrea, I d like to comment on this issue because we re going through a similar set of events. My employer has bought up several competing products and
              Message 6 of 6 , Feb 5, 2009
              View Source
              • 0 Attachment
                Hello Andrea,
                 
                I'd like to comment on this issue because we're going through a similar
                set of events.  My employer has bought up several competing products
                and has sent a new manager around looking for products that can be
                end of lifed to save money.
                 
                The products share nothing in common other than some of the tasks
                they perform.  Some similarities include the ability to initiate pages,
                send emails, and manage alarm systems.  All the product lines have
                an existing customer base, each of which may be offended by end-
                of-lifing a product that they are used to.
                 
                I brought this issue up with management because they didn't realize
                that they were doing useless work.  You see each of the product lines
                that have overlapping functionality are stable code bases.  There are
                enhancements elsewhere but no real ongoing maintenance.
                Combining the products by eliminating "paging" from one product
                in order to use the "paging" system of another was actually more
                work at this point.
                 
                Worse management bought up these competing lines, allowed
                development of all products to continue for a year and suddenly
                want to axe something.  They didn't notice that while the various
                products did compete against one another, there were clear
                preferences given by customers as to which products they
                wanted to have.  The actual overlap didn't amount to much.
                The customers have also rejected the notion that one product
                meets all potential customer needs.  Each product was created
                with a specific usability and functionality to address.  In the end,
                management has backed off the focus of actally axing products
                just to save costs.
                 
                We've entered a period where each of the groups have tried to
                cooperate more and management has started to see where each
                development group has something to offer the company as a
                whole.  Some groups are better at Agile Development, one
                is better at producing features at a low cost, and another has
                some moderate success at providing a better user experience.
                At this point managment has finally taken the approach that
                they want to look at what works and then make decisions.
                Blindly cutting off projects doesn't make sense in this case.
                 
                David


                Feb 3, 2009 03:09:13 PM, agile-usability@yahoogroups.com wrote:
                I am currently involved in a project to evaluate two similar products
                to determine which one should be end-of-lifed and which we should
                continue to sell. The products are web hosting platforms that
                customers access and managed via a gui interface.
                Of course this will be an extremely politically charged issue, thus I
                would like to use some recognized usability criteria for evaluation and
                ultimately move forward with the product that is easiest for our
                customers to use.
                I was thinking of creating some task based criteria. For example, how
                many steps does it take a user to create a simple website or add email
                user accounts. What other ways could i evaluate the products? Has
                anyone ever done this type of evaluation before?
                I don't want to get bogged down in how the information should be
                presented (because I out of my league on that type of evaluation) and
                will recommend that an expert take on that job.
                I look forward to any advice and guidance.
                Thank you,
                Andrea
              Your message has been successfully submitted and would be delivered to recipients shortly.