Loading ...
Sorry, an error occurred while loading the content.
 

Re: Leverage a small UX team?

Expand Messages
  • timkieschnick
    ... and the ... Has anyone done what Jon s suggesting? Somehow combinging usability test with the demo? I like the idea of showing live user testing to the
    Message 1 of 21 , Mar 3, 2008
      --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
      wrote:
      >
      > Would you run the usability test as part of the demo of the product
      > increment? I like the idea of the demo being the usability test
      and the
      > stakeholders being observers of the usability test.
      >

      Has anyone done what Jon's suggesting? Somehow combinging usability
      test with the demo?

      I like the idea of showing live user testing to the stakeholders,
      but it sounds problematic in terms of being able to show a
      representative sample of users to the stakeholders--I wouldn't want
      the stakeholders to see the one outlyer test, and I don't know how
      I'd show them 4-5 users in one demo.

      The only way I can think of to show them multiple users would be to
      splice together video clips of representative user tests. This would
      be fantastic in a demo, but too much work to do regularly.

      Anyone have experience with this?
    • Manish Pillewar
      From a sales/business perspective: Q. Would you run the usability test as part of the demo of the product increment? Comment: It has its risks. The users
      Message 2 of 21 , Mar 3, 2008
        From a sales/business perspective:

        Q. Would you run the usability test as part of the
        demo of the product increment?

        Comment: It has its risks. The users nominated for
        testing need to be very close to the persona's you
        created during the application design. Provided you
        have done all the risks mitigation, its should help
        drive the point home asap.


        Q. I like the idea of the demo being the usability
        test
        and the stakeholders being observers of the usability
        test.

        Comment: The challenge here is to moderate the test
        show while you bring out the best features of the
        product. Live usability testing may bring in
        additional viewpoints on how the client organization
        works and how certain additional features would have
        been great to have. This could potentially become a
        point of negotiation on the cost estimates for your
        product.


        Q. Has anyone done what Jon's suggesting? Somehow
        combining usability test with the demo? I like the
        idea of showing live user testing to the stakeholders,
        but it sounds problematic in terms of being able to
        show a representative sample of users to the
        stakeholders- -I wouldn't want the stakeholders to see
        the one outlyer test, and I don't know how I'd show
        them 4-5 users in one demo. The only way I can think
        of to show them multiple users would be to splice
        together video clips of representative user tests.
        This would be fantastic in a demo, but too much work
        to do regularly.Anyone have experience with this?

        Comment: Showing a short demo video coupled with the
        statistics generated from the usability test seems to
        be a better idea, in my experience. It not only shows
        that you have taken great care while designing the
        product,etc. but also generates less queries and
        lesser debates, as the data is for all to see.

        I've kept aside design ethics which would otherwise
        dictate that in case the user testing fails, its the
        feedback thats important, and you want to sell usable
        software only, even if it means that you don't make a
        sale at all.





        Thanks and Regards
        Manish Govind Pillewar
        Sr. User Experience Designer
        Thoughtworks India Pvt. Ltd.Bangalore-India

        Tel. +91 9880566951 (M)
        +91 80 41113967 (Eve.)
        Smith & Wesson: The original point and click interface :-)



        ___________________________________________________________
        Rise to the challenge for Sport Relief with Yahoo! For Good

        http://uk.promotions.yahoo.com/forgood/
      • Jeff Lopez-Stuit
        I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo
        Message 3 of 21 , Mar 3, 2008
          I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo the testing itself.   The objective of the demo should be to show what value was produced, not how the work was done.  You can provide a quick summary the tests, but the focus should be on the specific measurable improvements in usability that were identified and addressed during the sprint.

          Take care...

          Jeff Lopez-Stuit, CSM, CSPO
          SolutionsIQ

          On Mon, Mar 3, 2008 at 3:14 PM, timkieschnick <tim.kieschnick@...> wrote:

          --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
          wrote:
          >
          > Would you run the usability test as part of the demo of the product
          > increment? I like the idea of the demo being the usability test
          and the
          > stakeholders being observers of the usability test.
          >


          .


        • Jon Dickinson
          On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit ... I am making the assumption that the product owners already understand the value of usability testing,
          Message 4 of 21 , Mar 8, 2008
            On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit
            <jeff.lopezstuit@...> wrote:

            > I think it would be much more meaningful to the product owners if you
            > could
            > demonstrate the *value* of the testing you did during the sprint, rather
            > than demo the testing itself.

            I am making the assumption that the product owners already understand the
            value of usability testing, so it is not necessary to demonstrate the
            value of the technique.

            > The objective of the demo should be to show
            > what value was produced, not how the work was done.

            Agreed. The product managers attend a usability test not to see how a
            usability test is performed but to shorten the feedback loop from the
            results of the test.

            > You can provide a quick
            > summary the tests, but the focus should be on the specific measurable
            > improvements in usability that were identified and addressed during the
            > sprint.


            Over the course of a few end of sprint usability tests the product
            managers should get a good first hand impression of whether usability
            improvements are being made during the sprints. This doesn't stop one
            keeping records of the tests and producing measurements to show usability
            improvements over time, if this provides any additional value.

            --
            Jon Dickinson
            Accolade Consulting
            http://www.accolade-consulting.co.uk
          • Jon Dickinson
            On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar ... Is this not a useful exercise to validate your personas are correct? ... Assuming your product has more
            Message 5 of 21 , Mar 8, 2008
              On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
              <manish1022@...> wrote:

              >
              >
              >
              > From a sales/business perspective:
              >
              > Q. Would you run the usability test as part of the
              > demo of the product increment?
              >
              > Comment: It has its risks. The users nominated for
              > testing need to be very close to the persona's you
              > created during the application design.

              Is this not a useful exercise to validate your personas are correct?

              >
              > Q. I like the idea of the demo being the usability
              > test
              > and the stakeholders being observers of the usability
              > test.
              >
              > Comment: The challenge here is to moderate the test
              > show while you bring out the best features of the
              > product. Live usability testing may bring in
              > additional viewpoints on how the client organization
              > works and how certain additional features would have
              > been great to have. This could potentially become a
              > point of negotiation on the cost estimates for your
              > product.

              Assuming your product has more than one sprint, and therefore more than
              one usability test, before releasing why would this be a bad thing?

              > Comment: Showing a short demo video coupled with the
              > statistics generated from the usability test seems to
              > be a better idea, in my experience. It not only shows
              > that you have taken great care while designing the
              > product,etc. but also generates less queries and
              > lesser debates, as the data is for all to see.

              Why would you want to reduce debate? The idea of running a usibility test
              at the end of the sprint is generate discussion to help shape the work for
              the next sprint.

              --
              Jon Dickinson
              Accolade Consulting
              http://www.accolade-consulting.co.uk
            • Jon Dickinson
              On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick ... I can see that it would be a risk if the results of one bad test/demonstration to
              Message 6 of 21 , Mar 8, 2008
                On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick <tim.kieschnick@...>
                wrote:

                > I like the idea of showing live user testing to the stakeholders,
                > but it sounds problematic in terms of being able to show a
                > representative sample of users to the stakeholders--I wouldn't want
                > the stakeholders to see the one outlyer test, and I don't know how
                > I'd show them 4-5 users in one demo.

                I can see that it would be a risk if the results of one bad
                test/demonstration to the stakeholders sends the team off in the wrong
                direction for an entire sprint.

                From reading your excellent experience report:
                http://timiti.blogspot.com/2008/02/user-research-as-commodity.html, the
                following passage strikes me as providing two possible solutions:

                "Every other Thursday morning, four or five brokers (our target audience)
                would show up at our offices in Pasadena. Our user research specialist
                would work with each participant for about an hour. While this testing
                took place in Pasadena, we piped audio and video of the testing up north
                to Pleasanton so the team could watch in real-time and IM questions and
                comments to the moderator in Pasadena."

                1) Persuade the stakeholders to spend the entire morning at the end of the
                sprint observing the usability tests. This is my ideal.
                2) Have the stakeholders sit in as observers on the test with the last
                participant. If the results of this test turn out to be radically
                different to the earlier tests, you have the results of the previous tests
                to demonstrate that this was the exception.

                If the stakeholders are limited to observing a single test, it is probably
                worth giving them a half hour demo of the work completed before the
                usability test so they have an initial idea of what work has been
                completed. You don't want one bad test to negate all the hard work the
                team has performed during the sprint.

                Cheers,

                --
                Jon Dickinson
                Accolade Consulting
                http://www.accolade-consulting.co.uk
              • Manish Pillewar
                Hi Jon, Your questions are of course valid from a designers perspective.However,my comment have been from the sales/business perspective. refer the first
                Message 7 of 21 , Mar 9, 2008
                  Hi Jon,
                  Your questions are of course valid from a designers
                  perspective.However,my comment have been from the
                  sales/business perspective. refer the first statement
                  in my email

                  "> From a sales/business perspective:"


                  Hope you see that perspective.
                  -Manish


                  On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
                  <manish1022@...> wrote:

                  >
                  >
                  >
                  > From a sales/business perspective:
                  >
                  > Q. Would you run the usability test as part of the
                  > demo of the product increment?
                  >
                  > Comment: It has its risks. The users nominated for
                  > testing need to be very close to the persona's you
                  > created during the application design.

                  Is this not a useful exercise to validate your
                  personas are correct?

                  >
                  > Q. I like the idea of the demo being the usability
                  > test
                  > and the stakeholders being observers of the
                  usability
                  > test.
                  >
                  > Comment: The challenge here is to moderate the test
                  > show while you bring out the best features of the
                  > product. Live usability testing may bring in
                  > additional viewpoints on how the client organization
                  > works and how certain additional features would have
                  > been great to have. This could potentially become a
                  > point of negotiation on the cost estimates for your
                  > product.

                  Assuming your product has more than one sprint, and
                  therefore more than

                  one usability test, before releasing why would this be
                  a bad thing?

                  > Comment: Showing a short demo video coupled with the
                  > statistics generated from the usability test seems
                  to
                  > be a better idea, in my experience. It not only
                  shows
                  > that you have taken great care while designing the
                  > product,etc. but also generates less queries and
                  > lesser debates, as the data is for all to see.

                  Why would you want to reduce debate? The idea of
                  running a usibility
                  test
                  at the end of the sprint is generate discussion to
                  help shape the work
                  for
                  the next sprint.

                  --
                  Jon Dickinson
                  Accolade Consulting
                  http://www.accolade-consulting.co.uk

                  Thanks and Regards
                  Manish Govind Pillewar
                  Sr. User Experience Designer
                  Thoughtworks India Pvt. Ltd.Bangalore-India

                  Tel. +91 9880566951 (M)
                  +91 80 41113967 (Eve.)
                  Smith & Wesson: The original point and click interface :-)



                  __________________________________________________________
                  Sent from Yahoo! Mail.
                  The World's Favourite Email http://uk.docs.yahoo.com/nowyoucan.html
                • Jon Dickinson
                  On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar ... Hi Manesh, Are you talking about a situation where the business/sales goals are at odds with the goals
                  Message 8 of 21 , Mar 10, 2008
                    On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar
                    <manish1022@...> wrote:

                    > Hi Jon,
                    > Your questions are of course valid from a designers
                    > perspective.However,my comment have been from the
                    > sales/business perspective. refer the first statement
                    > in my email
                    >
                    > "> From a sales/business perspective:"
                    >
                    >
                    > Hope you see that perspective.
                    > -Manish
                    >

                    Hi Manesh,

                    Are you talking about a situation where the business/sales goals are at
                    odds with the goals of the rest of the team?

                    If that is the case then there is a bigger problem than trying to reduce
                    the feedback loop for usability testing. Although having stakeholders sit
                    in on the usability tests, but not as part of the end of sprint demo might
                    still help to address this.

                    If there is a misalignment between the goals of the stakeholders and the
                    implementation team then I would agree that this technique could cause
                    issues. There would need to be a certain amount of trust and an open
                    working relationship between the team and the stakeholders for this to
                    work.

                    Thanks,

                    --
                    Jon Dickinson
                    Accolade Consulting Ltd.
                    http://www.accolade-consulting.co.uk
                    http://www.planitagile.com
                  Your message has been successfully submitted and would be delivered to recipients shortly.