Loading ...
Sorry, an error occurred while loading the content.

User testing via 'out-of-context

Expand Messages
  • Maria
    Hi all, I d like to get your opinion on a question of weather it is appropriate to place something on a live site for sole purpose of gathering click-through
    Message 1 of 4 , Aug 15, 2011
    • 0 Attachment
      Hi all,

      I'd like to get your opinion on a question of weather it is appropriate to place something on a live site for sole purpose of gathering click-through data knowing that the placement of this particular link/feature will be out of context both in terms of content and style of the rest of the page.

      Here is the details of the problem. The design team I am leading is working on improving one of the key pages on the site. The page has overwhelming amount of content, features and lack of focus. We know from usability tests that users have trouble finding the content they are looking for. Some content takes too long to get to and we don't know if that content is important to our users.

      The lead of tech is proposing that we place "prominent links at the top of the page", one a time, that will lead users to pieces of this content. By placing a "large and bold link" at the top we will be able to tell, by click-through data if users are interested. These tests, she proposes, would be up for three days each.

      I am opposed to this type of testing as I believe that we would be making the page worse than it is now. Instead, I think that we would before successful in gaging users' interest in specific pieces of content in one-on-one user sessions. But since she is being adamant about this, I am loosing the number of arguments against her idea.

      What do you think?

      Maria
    • Georgi Varzonovtsev
      Maybe it would be a better option to do A/B testing or multivariate testing with Google website optimizer or similar tool. It allows you to present different
      Message 2 of 4 , Aug 15, 2011
      • 0 Attachment
        Maybe it would be a better option to do A/B testing or multivariate testing with Google website optimizer or similar tool. It allows you to present different versions of your page or part of page and compare who each version performed towards a predefined goal. It might depend on your CMS how difficult it is to implement but generally it shouldn't be a problem as it is as easy as embedding two or three lines of javascript into your pages.


        From: Maria <maria_gililova@...>
        To: agile-usability@yahoogroups.com
        Sent: Monday, August 15, 2011 4:37 PM
        Subject: [agile-usability] User testing via 'out-of-context

         
        Hi all,

        I'd like to get your opinion on a question of weather it is appropriate to place something on a live site for sole purpose of gathering click-through data knowing that the placement of this particular link/feature will be out of context both in terms of content and style of the rest of the page.

        Here is the details of the problem. The design team I am leading is working on improving one of the key pages on the site. The page has overwhelming amount of content, features and lack of focus. We know from usability tests that users have trouble finding the content they are looking for. Some content takes too long to get to and we don't know if that content is important to our users.

        The lead of tech is proposing that we place "prominent links at the top of the page", one a time, that will lead users to pieces of this content. By placing a "large and bold link" at the top we will be able to tell, by click-through data if users are interested. These tests, she proposes, would be up for three days each.

        I am opposed to this type of testing as I believe that we would be making the page worse than it is now. Instead, I think that we would before successful in gaging users' interest in specific pieces of content in one-on-one user sessions. But since she is being adamant about this, I am loosing the number of arguments against her idea.

        What do you think?

        Maria



      • Carol Smith
        Hello Maria, The A/B testing Georgi suggests would be a much better approach with one (or more) improved designs against the existing one. There is a risk
        Message 3 of 4 , Aug 15, 2011
        • 0 Attachment
          Hello Maria,

          The A/B testing Georgi suggests would be a much better approach with one (or more) improved designs against the existing one.  There is a risk with the linking idea that a user will expect those links to be there next time and be confused when they cannot find it.  Initially, I would suggest that some content be removed. While you don't want to hide content, it doesn't sound like they can find what's there now.  Little changes now are better than waiting for a perfect design later.

          Additionally, I would highly recommend that you do some user research. It doesn't have to be complicated or take up a lot of time. Quick ideas:
          - Go to a user's office for lunch and watch them use the site. 
          - Ask someone to meet you over GoToMeeting or Skype, share their screen, and watch them work and ask questions. 
          - Do a card-sorting exercise to organize content the way the user's expect them to be.
          - Do a survey of users and have them rank importance or prioritize information to inform design.

          Hope that helps!
          Carol

          -------------------

          Carol J. Smith

          Lead Consultant, Midwest Research, LLC

          http://www.mw-research.com


          On Mon, Aug 15, 2011 at 9:57 AM, Georgi Varzonovtsev <georgivar@...> wrote:
           

          Maybe it would be a better option to do A/B testing or multivariate testing with Google website optimizer or similar tool. It allows you to present different versions of your page or part of page and compare who each version performed towards a predefined goal. It might depend on your CMS how difficult it is to implement but generally it shouldn't be a problem as it is as easy as embedding two or three lines of javascript into your pages.


          From: Maria <maria_gililova@...>
          To: agile-usability@yahoogroups.com
          Sent: Monday, August 15, 2011 4:37 PM
          Subject: [agile-usability] User testing via 'out-of-context

           
          Hi all,

          I'd like to get your opinion on a question of weather it is appropriate to place something on a live site for sole purpose of gathering click-through data knowing that the placement of this particular link/feature will be out of context both in terms of content and style of the rest of the page.

          Here is the details of the problem. The design team I am leading is working on improving one of the key pages on the site. The page has overwhelming amount of content, features and lack of focus. We know from usability tests that users have trouble finding the content they are looking for. Some content takes too long to get to and we don't know if that content is important to our users.

          The lead of tech is proposing that we place "prominent links at the top of the page", one a time, that will lead users to pieces of this content. By placing a "large and bold link" at the top we will be able to tell, by click-through data if users are interested. These tests, she proposes, would be up for three days each.

          I am opposed to this type of testing as I believe that we would be making the page worse than it is now. Instead, I think that we would before successful in gaging users' interest in specific pieces of content in one-on-one user sessions. But since she is being adamant about this, I am loosing the number of arguments against her idea.

          What do you think?

          Maria




        • Maria Gililova
          Thank you, Carol and Georgi. I agree with the downside of adding the links and have, instead, been proposing A/B testing with an improved design as well. I
          Message 4 of 4 , Aug 15, 2011
          • 0 Attachment
            Thank you, Carol and Georgi.

            I agree with the downside of adding the links and have, instead, been proposing A/B testing with an improved design as well.
            I never thought of using Skype or GotoMeeting for usability testing. That's a great idea!

            Maria


            From: Carol Smith <carol@...>
            To: agile-usability@yahoogroups.com
            Sent: Monday, August 15, 2011 1:31 PM
            Subject: Re: [agile-usability] User testing via 'out-of-context

             
            Hello Maria,

            The A/B testing Georgi suggests would be a much better approach with one (or more) improved designs against the existing one.  There is a risk with the linking idea that a user will expect those links to be there next time and be confused when they cannot find it.  Initially, I would suggest that some content be removed. While you don't want to hide content, it doesn't sound like they can find what's there now.  Little changes now are better than waiting for a perfect design later.

            Additionally, I would highly recommend that you do some user research. It doesn't have to be complicated or take up a lot of time. Quick ideas:
            - Go to a user's office for lunch and watch them use the site. 
            - Ask someone to meet you over GoToMeeting or Skype, share their screen, and watch them work and ask questions. 
            - Do a card-sorting exercise to organize content the way the user's expect them to be.
            - Do a survey of users and have them rank importance or prioritize information to inform design.

            Hope that helps!
            Carol

            -------------------
            Carol J. Smith
            Lead Consultant, Midwest Research, LLC

            On Mon, Aug 15, 2011 at 9:57 AM, Georgi Varzonovtsev <georgivar@...> wrote:
             
            Maybe it would be a better option to do A/B testing or multivariate testing with Google website optimizer or similar tool. It allows you to present different versions of your page or part of page and compare who each version performed towards a predefined goal. It might depend on your CMS how difficult it is to implement but generally it shouldn't be a problem as it is as easy as embedding two or three lines of javascript into your pages.


            From: Maria <maria_gililova@...>
            To: agile-usability@yahoogroups.com
            Sent: Monday, August 15, 2011 4:37 PM
            Subject: [agile-usability] User testing via 'out-of-context

             
            Hi all,

            I'd like to get your opinion on a question of weather it is appropriate to place something on a live site for sole purpose of gathering click-through data knowing that the placement of this particular link/feature will be out of context both in terms of content and style of the rest of the page.

            Here is the details of the problem. The design team I am leading is working on improving one of the key pages on the site. The page has overwhelming amount of content, features and lack of focus. We know from usability tests that users have trouble finding the content they are looking for. Some content takes too long to get to and we don't know if that content is important to our users.

            The lead of tech is proposing that we place "prominent links at the top of the page", one a time, that will lead users to pieces of this content. By placing a "large and bold link" at the top we will be able to tell, by click-through data if users are interested. These tests, she proposes, would be up for three days each.

            I am opposed to this type of testing as I believe that we would be making the page worse than it is now. Instead, I think that we would before successful in gaging users' interest in specific pieces of content in one-on-one user sessions. But since she is being adamant about this, I am loosing the number of arguments against her idea.

            What do you think?

            Maria






          Your message has been successfully submitted and would be delivered to recipients shortly.