Loading ...
Sorry, an error occurred while loading the content.

Re: from qualitative user testing to quantitative user research

Expand Messages
  • Andy Edmonds
    Stefan: You d probably find this topic more productive on a different list, but there are some clear precedents for what you seek. For example, User Interface
    Message 1 of 6 , Oct 1, 2008
    • 0 Attachment
      Stefan: You'd probably find this topic more productive on a different
      list, but there are some clear precedents for what you seek.

      For example, User Interface Engineering (UIE), has done some nice high
      profile work across studies to show that longer links tend to perform
      better. IIRC, they correlated task success with link length across a
      bunch of studies. Read more at
      http://www.uie.com/reports/scent_of_information/

      In general, you're looking at "between subjects design" and a very
      large error component. You'll need a lot of data.

      The UIE example shows how you can apply a method of reducing each
      study to a datapoint and then analyzing across studies as an experiment.

      hth,
      Andy Edmonds
      http://surfmind.com

      --- In agile-usability@yahoogroups.com, "Stefan Wobben [Concept7]"
      <stefan@...> wrote:
      ...
      > I wonder if there is a way that i can gather data on a meta-level so
      in the end we are able to perform statitic analyses. The problem
      however is we there are different websites, different tasks and
      different participants.
      ...
      > I can't believe there isn't any other possibility. Does anybody has
      ideas? and/or interested in performing such type of studies?
    • James Page
      ... overview of which method to use. Designed for medical researches but most of methods apply to usability. Also look at the website
      Message 2 of 6 , Oct 1, 2008
      • 0 Attachment

        Get hold of a copy of "Statistics in Medicine". It gives you a good overview of which method to use. Designed for medical researches but most of methods apply to usability. 
         
        Also look at the website http://www.measuringusability.com/ for a good overview of statistics and usability. 

        I would agree that you need lots of data. 

        James


         
        On Wed, Oct 1, 2008 at 11:41 AM, Andy Edmonds <andyed@...> wrote:

        Stefan: You'd probably find this topic more productive on a different
        list, but there are some clear precedents for what you seek.

        For example, User Interface Engineering (UIE), has done some nice high
        profile work across studies to show that longer links tend to perform
        better. IIRC, they correlated task success with link length across a
        bunch of studies. Read more at
        http://www.uie.com/reports/scent_of_information/

        In general, you're looking at "between subjects design" and a very
        large error component. You'll need a lot of data.

        The UIE example shows how you can apply a method of reducing each
        study to a datapoint and then analyzing across studies as an experiment.

        hth,
        Andy Edmonds
        http://surfmind.com

        --- In agile-usability@yahoogroups.com, "Stefan Wobben [Concept7]"
        <stefan@...> wrote:
        ...


        > I wonder if there is a way that i can gather data on a meta-level so
        in the end we are able to perform statitic analyses. The problem
        however is we there are different websites, different tasks and
        different participants.
        ...

        > I can't believe there isn't any other possibility. Does anybody has
        ideas? and/or interested in performing such type of studies?


      • Stefan
        Thank you Andy. Thank you James. I ve ordered scent of information and statistics in medicine so i can some research on the topic. As the remark you need lots
        Message 3 of 6 , Oct 1, 2008
        • 0 Attachment
          Thank you Andy.
          Thank you James.

          I've ordered scent of information and statistics in medicine so i can some research on the
          topic.

          As the remark 'you need lots of data' well thats what we have :)

          I'll keep you updated about my progress.
        • Jared Spool
          ... I hope you won t be disappointed with the Scent of Information report. It doesn t really go into our methodology. That s not why we created it. What you re
          Message 4 of 6 , Oct 1, 2008
          • 0 Attachment
            On Oct 1, 2008, at 4:09 PM, Stefan wrote:

            > I've ordered scent of information and statistics in medicine so i
            > can some research on the
            > topic.

            I hope you won't be disappointed with the Scent of Information report.
            It doesn't really go into our methodology. That's not why we created it.

            What you're trying to do is *really* hard and will take huge amounts
            of preparation and some massively impressive statistics horsepower. I
            know because we've tried multiple times and only managed to pull it
            off some of the time.

            There are scaling and ground issues. Ground issues are about
            connecting the independent data collection points to a common element,
            so you can say that these measures are talking about the same thing.
            Scaling issues are about ensuring that Good/Bad in one scale mean the
            same as Good/Bad in another scale.

            This is very heavy duty science you're trying to do. I'm betting, if
            you want to do it well, it will triple-to-quintuple your research costs.

            I'm betting that's why you've not found a lot of resources on it. You
            need a deep pocket to make it really work.

            Jared

            Jared M. Spool
            User Interface Engineering
            510 Turnpike St., Suite 102, North Andover, MA 01845
            e: jspool@... p: +1 978 327 5561
            http://uie.com Blog: http://uie.com/brainsparks
          • Stefan
            Jared, Have you read the usability problem taxonomy ? of Keenan, Hartson, Kafura and Schulman? What do you think of using this classification scheme for
            Message 5 of 6 , Oct 12, 2008
            • 0 Attachment
              Jared,

              Have you read 'the usability problem taxonomy'? of Keenan, Hartson, Kafura and
              Schulman? What do you think of using this classification scheme for between-subjects
              studies?

              Stefan

              --- In agile-usability@yahoogroups.com, Jared Spool <jspool@...> wrote:
              >
              >
              > On Oct 1, 2008, at 4:09 PM, Stefan wrote:
              >
              > > I've ordered scent of information and statistics in medicine so i
              > > can some research on the
              > > topic.
              >
              > I hope you won't be disappointed with the Scent of Information report.
              > It doesn't really go into our methodology. That's not why we created it.
              >
              > What you're trying to do is *really* hard and will take huge amounts
              > of preparation and some massively impressive statistics horsepower. I
              > know because we've tried multiple times and only managed to pull it
              > off some of the time.
              >
              > There are scaling and ground issues. Ground issues are about
              > connecting the independent data collection points to a common element,
              > so you can say that these measures are talking about the same thing.
              > Scaling issues are about ensuring that Good/Bad in one scale mean the
              > same as Good/Bad in another scale.
              >
              > This is very heavy duty science you're trying to do. I'm betting, if
              > you want to do it well, it will triple-to-quintuple your research costs.
              >
              > I'm betting that's why you've not found a lot of resources on it. You
              > need a deep pocket to make it really work.
              >
              > Jared
              >
              > Jared M. Spool
              > User Interface Engineering
              > 510 Turnpike St., Suite 102, North Andover, MA 01845
              > e: jspool@... p: +1 978 327 5561
              > http://uie.com Blog: http://uie.com/brainsparks
              >
            Your message has been successfully submitted and would be delivered to recipients shortly.