Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Leverage a small UX team?

Expand Messages
  • Adrian Howard
    On 31 Jan 2008, at 09:45, Manish Pillewar wrote: [snip] ... [snip] Not sure that I d describe it as perfect :-) I find that you get a _lot_ more value by
    Message 1 of 21 , Feb 1, 2008
    • 0 Attachment
      On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
      [snip]
      > >> 3) We're thinking that the best times to bring in the UX
      > specialists are
      > >> (a) project inception, prior to forming the team, to help guide
      > initial
      > >> user research (ethnography, persona development, etc.) and (b)
      > during
      > >> release planning (or maybe between each sprint), to help the
      > team ID
      > >> what tools from the world of user-centered design will be most
      > helpful
      > >> for the coming work, and how the UX team can support that.
      >
      > This is perfect. You can also choose to involve the designers for
      > POC activities or use their creativity to conceptulize UI solutions
      > during sales pitches or/and do competitive benchmarking exercises
      > as a showcase of Ux capabilities. QA is one more arena where the
      > designer can help in. She can do design reviews or usability
      > evaluations( heuristics,etc), even quick usability testing on the
      > developed screens at every iteration end, to ensure that the UX
      > standards are maintained.

      [snip]

      Not sure that I'd describe it as perfect :-)

      I find that you get a _lot_ more value by having UX folk involved
      with the team as close to 100% as possible. While there is obviously
      a lot "big" UX work that sits on the customer side, communicating it
      is much better done in person than with documents. There are also
      always UX issues that pop up during development. Having a UX person
      on-hand makes sure that they'll be addressed in an appropriate manner.

      So my perfect set up has at least one UX person as a member of the
      team 100% of the time.

      Which of course sucks as a solution if, like the OP, you only fewer
      UX folk than development teams...

      Assuming that hiring a bunch more UX dudes isn't practical, I'd be
      looking at spending a lot more time in general education of the whole
      team in UX issues. That way there's more chance that UX issues that
      pop up mid-iteration are handled well (even/especially when the
      method is "get a UX dude here stat" :-)

      Cheers,

      Adrian
    • timkieschnick
      Thanks for the good advice. I m clearly not going to get another dozen UX specialists, so I need to leverage my 4 specialists across the people in the
      Message 2 of 21 , Feb 4, 2008
      • 0 Attachment
        Thanks for the good advice. I'm clearly not going to get another
        dozen UX specialists, so I need to leverage my 4 specialists across
        the people in the dedicated teams. Here's what I'm thinking in terms
        of training priorities:

        First, Product Owners:
        - evangelize them on user experience
        - introduce them to the core toolset and the support available
        - think together about how to build UX into the product backlog and
        sprint exit criteria
        - work with them on an ongoing shared product backlog for things
        like creating and implementing design patterns or templates across
        the site

        Second, Business Analysts:
        - evangelize--give them something to live for to replace the
        requirements documents they've been lost in for the last 3 years
        - introduce them to the core toolset and the support available
        - give them the hands-on skills to do things like user-centered
        stories, effective use of personas, and card sorts

        Third, Everybody Else:
        At this point I'll have a skeleton of a support system in place, and
        two people on each team who can tangibly make use of general
        excitement about the user experience.
        - evangelize
        - UX and the product lifecylce
        - when to call for help

        Fourth, Scrummasters:
        - collaborative working session to come up with some shared best
        practices and places to innovate

        Having not done scrum yet myself, I'm probably front-loading too
        much. There will certainly be plenty of informal alignment &
        education going on, but how does this look for the formal training
        part?


        --- In agile-usability@yahoogroups.com, Adrian Howard <adrianh@...>
        wrote:
        >
        > On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
        > [snip]
        > > >> 3) We're thinking that the best times to bring in the UX
        > > specialists are
        > > >> (a) project inception, prior to forming the team, to help
        guide
        > > initial
        > > >> user research (ethnography, persona development, etc.) and
        (b)
        > > during
        > > >> release planning (or maybe between each sprint), to help
        the
        > > team ID
        > > >> what tools from the world of user-centered design will be
        most
        > > helpful
        > > >> for the coming work, and how the UX team can support that.
        > >
        > > This is perfect. You can also choose to involve the designers
        for
        > > POC activities or use their creativity to conceptulize UI
        solutions
        > > during sales pitches or/and do competitive benchmarking
        exercises
        > > as a showcase of Ux capabilities. QA is one more arena where
        the
        > > designer can help in. She can do design reviews or usability
        > > evaluations( heuristics,etc), even quick usability testing on
        the
        > > developed screens at every iteration end, to ensure that the UX
        > > standards are maintained.
        >
        > [snip]
        >
        > Not sure that I'd describe it as perfect :-)
        >
        > I find that you get a _lot_ more value by having UX folk involved
        > with the team as close to 100% as possible. While there is
        obviously
        > a lot "big" UX work that sits on the customer side, communicating
        it
        > is much better done in person than with documents. There are also
        > always UX issues that pop up during development. Having a UX
        person
        > on-hand makes sure that they'll be addressed in an appropriate
        manner.
        >
        > So my perfect set up has at least one UX person as a member of
        the
        > team 100% of the time.
        >
        > Which of course sucks as a solution if, like the OP, you only
        fewer
        > UX folk than development teams...
        >
        > Assuming that hiring a bunch more UX dudes isn't practical, I'd
        be
        > looking at spending a lot more time in general education of the
        whole
        > team in UX issues. That way there's more chance that UX issues
        that
        > pop up mid-iteration are handled well (even/especially when the
        > method is "get a UX dude here stat" :-)
        >
        > Cheers,
        >
        > Adrian
        >
      • Jon Dickinson
        I would be tempted to move Scrum Masters higher on the list. While I think you are right to target product owners first, as they should get the final say on
        Message 3 of 21 , Feb 4, 2008
        • 0 Attachment
          I would be tempted to move Scrum Masters higher on the list. While I think you are right to target product owners first, as they should get the final say on what goes into their product, Scrum Masters should be working closely with the product owners so they have an idea of what is coming up for the next sprint. These people also have the respect of the development team as it is their job to make the teams life easier. It will be useful to build relationships with Scrum Masters early on to figure out how you can fit the UX activities into the sprint cycle.

          I like your idea of building UX into the sprint exit criteria. Do you have any ideas how you are going to achieve this yet?

          Thanks,
          Jon.
          http://www.accolade-consulting.co.uk

          On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:

          Thanks for the good advice. I'm clearly not going to get another
          dozen UX specialists, so I need to leverage my 4 specialists across
          the people in the dedicated teams. Here's what I'm thinking in terms
          of training priorities:

          First, Product Owners:
          - evangelize them on user experience
          - introduce them to the core toolset and the support available
          - think together about how to build UX into the product backlog and
          sprint exit criteria
          - work with them on an ongoing shared product backlog for things
          like creating and implementing design patterns or templates across
          the site

          Second, Business Analysts:
          - evangelize--give them something to live for to replace the
          requirements documents they've been lost in for the last 3 years
          - introduce them to the core toolset and the support available
          - give them the hands-on skills to do things like user-centered
          stories, effective use of personas, and card sorts

          Third, Everybody Else:
          At this point I'll have a skeleton of a support system in place, and
          two people on each team who can tangibly make use of general
          excitement about the user experience.
          - evangelize
          - UX and the product lifecylce
          - when to call for help

          Fourth, Scrummasters:
          - collaborative working session to come up with some shared best
          practices and places to innovate

          Having not done scrum yet myself, I'm probably front-loading too
          much. There will certainly be plenty of informal alignment &
          education going on, but how does this look for the formal training
          part?

          --- In agile-usability@yahoogroups.com, Adrian Howard <adrianh@...>


          wrote:
          >
          > On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
          > [snip]
          > > >> 3) We're thinking that the best times to bring in the UX
          > > specialists are
          > > >> (a) project inception, prior to forming the team, to help
          guide
          > > initial
          > > >> user research (ethnography, persona development, etc.) and
          (b)
          > > during
          > > >> release planning (or maybe between each sprint), to help
          the
          > > team ID
          > > >> what tools from the world of user-centered design will be
          most
          > > helpful
          > > >> for the coming work, and how the UX team can support that.
          > >
          > > This is perfect. You can also choose to involve the designers
          for
          > > POC activities or use their creativity to conceptulize UI
          solutions
          > > during sales pitches or/and do competitive benchmarking
          exercises
          > > as a showcase of Ux capabilities. QA is one more arena where
          the
          > > designer can help in. She can do design reviews or usability
          > > evaluations( heuristics,etc), even quick usability testing on
          the
          > > developed screens at every iteration end, to ensure that the UX
          > > standards are maintained.
          >
          > [snip]
          >
          > Not sure that I'd describe it as perfect :-)
          >
          > I find that you get a _lot_ more value by having UX folk involved
          > with the team as close to 100% as possible. While there is
          obviously
          > a lot "big" UX work that sits on the customer side, communicating
          it
          > is much better done in person than with documents. There are also
          > always UX issues that pop up during development. Having a UX
          person
          > on-hand makes sure that they'll be addressed in an appropriate
          manner.
          >
          > So my perfect set up has at least one UX person as a member of
          the
          > team 100% of the time.
          >
          > Which of course sucks as a solution if, like the OP, you only
          fewer
          > UX folk than development teams...
          >
          > Assuming that hiring a bunch more UX dudes isn't practical, I'd
          be
          > looking at spending a lot more time in general education of the
          whole
          > team in UX issues. That way there's more chance that UX issues
          that
          > pop up mid-iteration are handled well (even/especially when the
          > method is "get a UX dude here stat" :-)
          >
          > Cheers,
          >
          > Adrian
          >


        • timkieschnick
          ... you have ... My ideas so far: 1) Be sure exit criteria is framed from the user s perspective, e.g. Katherine (our primary person) can cancel an order.
          Message 4 of 21 , Feb 4, 2008
          • 0 Attachment
            --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
            wrote:
            > I like your idea of building UX into the sprint exit criteria. Do
            you have
            > any ideas how you are going to achieve this yet?

            My ideas so far:

            1) Be sure exit criteria is framed from the user's perspective,
            e.g. "Katherine (our primary person) can cancel an order." rather
            than "implement cancel order functionality"

            2) I think there may be some instances where it will make sense to
            put a usability metric on the exit criteria, e.g., "Users can cancel
            an order, and they rate their experience an average of at least 4 on
            either the SUS or the question, "Compared to other sites, how easy
            was this to use?"

            This metric would be supported by what I'm calling "commoditized
            user research"--this is something that we've started doing that sort
            of turns most user research into a commodity--it makes it very
            simple for the project team to get user feedback. I can say much
            more on that if anyone's interested.
          • eric.idebro
            ... I can say much more on that if anyone s interested. ... Please do!
            Message 5 of 21 , Feb 4, 2008
            • 0 Attachment
              >
              > My ideas so far:
              I can say much more on that if anyone's interested.

              >







              Please do!
            • Jon Dickinson
              ... Have you considered a team based approach to training rather than role based?
              Message 6 of 21 , Feb 5, 2008
              • 0 Attachment
                On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:


                Having not done scrum yet myself, I'm probably front-loading too
                much. There will certainly be plenty of informal alignment &
                education going on, but how does this look for the formal training
                part?












                Have you considered a team based approach to training rather than role based?
              • Jon Dickinson
                Would you run the usability test as part of the demo of the product increment? I like the idea of the demo being the usability test and the stakeholders being
                Message 7 of 21 , Feb 5, 2008
                • 0 Attachment
                  Would you run the usability test as part of the demo of the product increment? I like the idea of the demo being the usability test and the stakeholders being observers of the usability test.

                  > This metric would be supported by what I'm calling "commoditized
                  > user research"

                  Sounds like a blog post in the making. I would write this up outside of the list and then send a link to the list. I'm sure it would be a great way to get feedback. Not that I'm a list administrator, I've just seen people do this in the XP list and it works very well.

                  Jon.




                  On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:

                  --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                  wrote:
                  > I like your idea of building UX into the sprint exit criteria. Do
                  you have
                  > any ideas how you are going to achieve this yet?

                  My ideas so far:

                  1) Be sure exit criteria is framed from the user's perspective,
                  e.g. "Katherine (our primary person) can cancel an order." rather
                  than "implement cancel order functionality"

                  2) I think there may be some instances where it will make sense to
                  put a usability metric on the exit criteria, e.g., "Users can cancel
                  an order, and they rate their experience an average of at least 4 on
                  either the SUS or the question, "Compared to other sites, how easy
                  was this to use?"

                  This metric would be supported by what I'm calling "commoditized
                  user research"--this is something that we've started doing that sort
                  of turns most user research into a commodity--it makes it very
                  simple for the project team to get user feedback. I can say much
                  more on that if anyone's interested.


                • Adrian Howard
                  ... That was what I was just about to suggest. Once you have an organisation that accepts that UX issues are important (and if they re financing a UX
                  Message 8 of 21 , Feb 5, 2008
                  • 0 Attachment
                    On 5 Feb 2008, at 09:04, Jon Dickinson wrote:

                    >> On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:
                    >>
                    >>>
                    >>> Having not done scrum yet myself, I'm probably front-loading too
                    >>> much. There will certainly be plenty of informal alignment &
                    >>> education going on, but how does this look for the formal training
                    >>> part?
                    >
                    > Have you considered a team based approach to training rather than role
                    > based?

                    That was what I was just about to suggest. Once you have an
                    organisation that accepts that UX issues are important (and if
                    they're financing a UX department than my guess is the answer to that
                    is yes :-) then I'd focus on educating project teams as a whole
                    rather than roles.

                    In my experience UX in agile teams is as much bottom-up as it is top-
                    down (like most other things in agile teams). I think you'll see see
                    more progress more quickly if you educate a team at a time, rather
                    than a layer that cuts across all teams.

                    Cheers,

                    Adrian
                  • Adrian Howard
                    ... [snip] I ve only managed to do it a couple of times but I find whole-team usability tests very useful. While he s not talking about an agile environment
                    Message 9 of 21 , Feb 5, 2008
                    • 0 Attachment
                      On 5 Feb 2008, at 09:53, Jon Dickinson wrote:

                      > Would you run the usability test as part of the demo of the product
                      > increment? I like the idea of the demo being the usability test and
                      > the stakeholders being observers of the usability test.
                      [snip]

                      I've only managed to do it a couple of times but I find whole-team
                      usability tests very useful. While he's not talking about an agile
                      environment Jared Spool has a nice bit on testing with large groups
                      in these two blog posts:

                      * http://www.uie.com/brainsparks/2007/09/13/usability-tests-with-30-
                      observers/
                      * http://www.uie.com/brainsparks/2007/09/18/more-on-usability-tests-
                      with-30-observers/

                      Cheers,

                      Adrian
                    • Adrian Howard
                      On 4 Feb 2008, at 22:59, timkieschnick wrote: [snip] ... [snip] While I m not a huge metric fan, if you re having problem convincing folk about the importance
                      Message 10 of 21 , Feb 5, 2008
                      • 0 Attachment
                        On 4 Feb 2008, at 22:59, timkieschnick wrote:
                        [snip]
                        > 2) I think there may be some instances where it will make sense to
                        > put a usability metric on the exit criteria, e.g., "Users can cancel
                        > an order, and they rate their experience an average of at least 4 on
                        > either the SUS or the question, "Compared to other sites, how easy
                        > was this to use?"
                        [snip]

                        While I'm not a huge metric fan, if you're having problem convincing
                        folk about the importance of UX issues having a graph of SUS numbers
                        over time in the wall along with the burndown chart, et al, can be
                        useful :-)

                        Cheers,

                        Adrian
                      • Gart, Mitchell
                        In our company we have a few UX people spread over several development teams. Each UX person has to split their time between a few development projects. Each
                        Message 11 of 21 , Feb 5, 2008
                        • 0 Attachment
                          In our company we have a few UX people spread over several development
                          teams. Each UX person has to split their time between a few development
                          projects. Each team has several full time developers and one part time
                          UX person.

                          I find that the most important thing is having a good, trusting working
                          relationship. If the developers trust the UX person, and vice versa,
                          they will be able to work together with this kind of full time - part
                          time division of labor. If they don't have a trusting relationship,
                          things will not go so well.

                          - Mitch Gart
                        • timkieschnick
                          ... and the ... Has anyone done what Jon s suggesting? Somehow combinging usability test with the demo? I like the idea of showing live user testing to the
                          Message 12 of 21 , Mar 3, 2008
                          • 0 Attachment
                            --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                            wrote:
                            >
                            > Would you run the usability test as part of the demo of the product
                            > increment? I like the idea of the demo being the usability test
                            and the
                            > stakeholders being observers of the usability test.
                            >

                            Has anyone done what Jon's suggesting? Somehow combinging usability
                            test with the demo?

                            I like the idea of showing live user testing to the stakeholders,
                            but it sounds problematic in terms of being able to show a
                            representative sample of users to the stakeholders--I wouldn't want
                            the stakeholders to see the one outlyer test, and I don't know how
                            I'd show them 4-5 users in one demo.

                            The only way I can think of to show them multiple users would be to
                            splice together video clips of representative user tests. This would
                            be fantastic in a demo, but too much work to do regularly.

                            Anyone have experience with this?
                          • Manish Pillewar
                            From a sales/business perspective: Q. Would you run the usability test as part of the demo of the product increment? Comment: It has its risks. The users
                            Message 13 of 21 , Mar 3, 2008
                            • 0 Attachment
                              From a sales/business perspective:

                              Q. Would you run the usability test as part of the
                              demo of the product increment?

                              Comment: It has its risks. The users nominated for
                              testing need to be very close to the persona's you
                              created during the application design. Provided you
                              have done all the risks mitigation, its should help
                              drive the point home asap.


                              Q. I like the idea of the demo being the usability
                              test
                              and the stakeholders being observers of the usability
                              test.

                              Comment: The challenge here is to moderate the test
                              show while you bring out the best features of the
                              product. Live usability testing may bring in
                              additional viewpoints on how the client organization
                              works and how certain additional features would have
                              been great to have. This could potentially become a
                              point of negotiation on the cost estimates for your
                              product.


                              Q. Has anyone done what Jon's suggesting? Somehow
                              combining usability test with the demo? I like the
                              idea of showing live user testing to the stakeholders,
                              but it sounds problematic in terms of being able to
                              show a representative sample of users to the
                              stakeholders- -I wouldn't want the stakeholders to see
                              the one outlyer test, and I don't know how I'd show
                              them 4-5 users in one demo. The only way I can think
                              of to show them multiple users would be to splice
                              together video clips of representative user tests.
                              This would be fantastic in a demo, but too much work
                              to do regularly.Anyone have experience with this?

                              Comment: Showing a short demo video coupled with the
                              statistics generated from the usability test seems to
                              be a better idea, in my experience. It not only shows
                              that you have taken great care while designing the
                              product,etc. but also generates less queries and
                              lesser debates, as the data is for all to see.

                              I've kept aside design ethics which would otherwise
                              dictate that in case the user testing fails, its the
                              feedback thats important, and you want to sell usable
                              software only, even if it means that you don't make a
                              sale at all.





                              Thanks and Regards
                              Manish Govind Pillewar
                              Sr. User Experience Designer
                              Thoughtworks India Pvt. Ltd.Bangalore-India

                              Tel. +91 9880566951 (M)
                              +91 80 41113967 (Eve.)
                              Smith & Wesson: The original point and click interface :-)



                              ___________________________________________________________
                              Rise to the challenge for Sport Relief with Yahoo! For Good

                              http://uk.promotions.yahoo.com/forgood/
                            • Jeff Lopez-Stuit
                              I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo
                              Message 14 of 21 , Mar 3, 2008
                              • 0 Attachment
                                I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo the testing itself.   The objective of the demo should be to show what value was produced, not how the work was done.  You can provide a quick summary the tests, but the focus should be on the specific measurable improvements in usability that were identified and addressed during the sprint.

                                Take care...

                                Jeff Lopez-Stuit, CSM, CSPO
                                SolutionsIQ

                                On Mon, Mar 3, 2008 at 3:14 PM, timkieschnick <tim.kieschnick@...> wrote:

                                --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                                wrote:
                                >
                                > Would you run the usability test as part of the demo of the product
                                > increment? I like the idea of the demo being the usability test
                                and the
                                > stakeholders being observers of the usability test.
                                >


                                .


                              • Jon Dickinson
                                On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit ... I am making the assumption that the product owners already understand the value of usability testing,
                                Message 15 of 21 , Mar 8, 2008
                                • 0 Attachment
                                  On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit
                                  <jeff.lopezstuit@...> wrote:

                                  > I think it would be much more meaningful to the product owners if you
                                  > could
                                  > demonstrate the *value* of the testing you did during the sprint, rather
                                  > than demo the testing itself.

                                  I am making the assumption that the product owners already understand the
                                  value of usability testing, so it is not necessary to demonstrate the
                                  value of the technique.

                                  > The objective of the demo should be to show
                                  > what value was produced, not how the work was done.

                                  Agreed. The product managers attend a usability test not to see how a
                                  usability test is performed but to shorten the feedback loop from the
                                  results of the test.

                                  > You can provide a quick
                                  > summary the tests, but the focus should be on the specific measurable
                                  > improvements in usability that were identified and addressed during the
                                  > sprint.


                                  Over the course of a few end of sprint usability tests the product
                                  managers should get a good first hand impression of whether usability
                                  improvements are being made during the sprints. This doesn't stop one
                                  keeping records of the tests and producing measurements to show usability
                                  improvements over time, if this provides any additional value.

                                  --
                                  Jon Dickinson
                                  Accolade Consulting
                                  http://www.accolade-consulting.co.uk
                                • Jon Dickinson
                                  On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar ... Is this not a useful exercise to validate your personas are correct? ... Assuming your product has more
                                  Message 16 of 21 , Mar 8, 2008
                                  • 0 Attachment
                                    On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
                                    <manish1022@...> wrote:

                                    >
                                    >
                                    >
                                    > From a sales/business perspective:
                                    >
                                    > Q. Would you run the usability test as part of the
                                    > demo of the product increment?
                                    >
                                    > Comment: It has its risks. The users nominated for
                                    > testing need to be very close to the persona's you
                                    > created during the application design.

                                    Is this not a useful exercise to validate your personas are correct?

                                    >
                                    > Q. I like the idea of the demo being the usability
                                    > test
                                    > and the stakeholders being observers of the usability
                                    > test.
                                    >
                                    > Comment: The challenge here is to moderate the test
                                    > show while you bring out the best features of the
                                    > product. Live usability testing may bring in
                                    > additional viewpoints on how the client organization
                                    > works and how certain additional features would have
                                    > been great to have. This could potentially become a
                                    > point of negotiation on the cost estimates for your
                                    > product.

                                    Assuming your product has more than one sprint, and therefore more than
                                    one usability test, before releasing why would this be a bad thing?

                                    > Comment: Showing a short demo video coupled with the
                                    > statistics generated from the usability test seems to
                                    > be a better idea, in my experience. It not only shows
                                    > that you have taken great care while designing the
                                    > product,etc. but also generates less queries and
                                    > lesser debates, as the data is for all to see.

                                    Why would you want to reduce debate? The idea of running a usibility test
                                    at the end of the sprint is generate discussion to help shape the work for
                                    the next sprint.

                                    --
                                    Jon Dickinson
                                    Accolade Consulting
                                    http://www.accolade-consulting.co.uk
                                  • Jon Dickinson
                                    On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick ... I can see that it would be a risk if the results of one bad test/demonstration to
                                    Message 17 of 21 , Mar 8, 2008
                                    • 0 Attachment
                                      On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick <tim.kieschnick@...>
                                      wrote:

                                      > I like the idea of showing live user testing to the stakeholders,
                                      > but it sounds problematic in terms of being able to show a
                                      > representative sample of users to the stakeholders--I wouldn't want
                                      > the stakeholders to see the one outlyer test, and I don't know how
                                      > I'd show them 4-5 users in one demo.

                                      I can see that it would be a risk if the results of one bad
                                      test/demonstration to the stakeholders sends the team off in the wrong
                                      direction for an entire sprint.

                                      From reading your excellent experience report:
                                      http://timiti.blogspot.com/2008/02/user-research-as-commodity.html, the
                                      following passage strikes me as providing two possible solutions:

                                      "Every other Thursday morning, four or five brokers (our target audience)
                                      would show up at our offices in Pasadena. Our user research specialist
                                      would work with each participant for about an hour. While this testing
                                      took place in Pasadena, we piped audio and video of the testing up north
                                      to Pleasanton so the team could watch in real-time and IM questions and
                                      comments to the moderator in Pasadena."

                                      1) Persuade the stakeholders to spend the entire morning at the end of the
                                      sprint observing the usability tests. This is my ideal.
                                      2) Have the stakeholders sit in as observers on the test with the last
                                      participant. If the results of this test turn out to be radically
                                      different to the earlier tests, you have the results of the previous tests
                                      to demonstrate that this was the exception.

                                      If the stakeholders are limited to observing a single test, it is probably
                                      worth giving them a half hour demo of the work completed before the
                                      usability test so they have an initial idea of what work has been
                                      completed. You don't want one bad test to negate all the hard work the
                                      team has performed during the sprint.

                                      Cheers,

                                      --
                                      Jon Dickinson
                                      Accolade Consulting
                                      http://www.accolade-consulting.co.uk
                                    • Manish Pillewar
                                      Hi Jon, Your questions are of course valid from a designers perspective.However,my comment have been from the sales/business perspective. refer the first
                                      Message 18 of 21 , Mar 9, 2008
                                      • 0 Attachment
                                        Hi Jon,
                                        Your questions are of course valid from a designers
                                        perspective.However,my comment have been from the
                                        sales/business perspective. refer the first statement
                                        in my email

                                        "> From a sales/business perspective:"


                                        Hope you see that perspective.
                                        -Manish


                                        On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
                                        <manish1022@...> wrote:

                                        >
                                        >
                                        >
                                        > From a sales/business perspective:
                                        >
                                        > Q. Would you run the usability test as part of the
                                        > demo of the product increment?
                                        >
                                        > Comment: It has its risks. The users nominated for
                                        > testing need to be very close to the persona's you
                                        > created during the application design.

                                        Is this not a useful exercise to validate your
                                        personas are correct?

                                        >
                                        > Q. I like the idea of the demo being the usability
                                        > test
                                        > and the stakeholders being observers of the
                                        usability
                                        > test.
                                        >
                                        > Comment: The challenge here is to moderate the test
                                        > show while you bring out the best features of the
                                        > product. Live usability testing may bring in
                                        > additional viewpoints on how the client organization
                                        > works and how certain additional features would have
                                        > been great to have. This could potentially become a
                                        > point of negotiation on the cost estimates for your
                                        > product.

                                        Assuming your product has more than one sprint, and
                                        therefore more than

                                        one usability test, before releasing why would this be
                                        a bad thing?

                                        > Comment: Showing a short demo video coupled with the
                                        > statistics generated from the usability test seems
                                        to
                                        > be a better idea, in my experience. It not only
                                        shows
                                        > that you have taken great care while designing the
                                        > product,etc. but also generates less queries and
                                        > lesser debates, as the data is for all to see.

                                        Why would you want to reduce debate? The idea of
                                        running a usibility
                                        test
                                        at the end of the sprint is generate discussion to
                                        help shape the work
                                        for
                                        the next sprint.

                                        --
                                        Jon Dickinson
                                        Accolade Consulting
                                        http://www.accolade-consulting.co.uk

                                        Thanks and Regards
                                        Manish Govind Pillewar
                                        Sr. User Experience Designer
                                        Thoughtworks India Pvt. Ltd.Bangalore-India

                                        Tel. +91 9880566951 (M)
                                        +91 80 41113967 (Eve.)
                                        Smith & Wesson: The original point and click interface :-)



                                        __________________________________________________________
                                        Sent from Yahoo! Mail.
                                        The World's Favourite Email http://uk.docs.yahoo.com/nowyoucan.html
                                      • Jon Dickinson
                                        On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar ... Hi Manesh, Are you talking about a situation where the business/sales goals are at odds with the goals
                                        Message 19 of 21 , Mar 10, 2008
                                        • 0 Attachment
                                          On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar
                                          <manish1022@...> wrote:

                                          > Hi Jon,
                                          > Your questions are of course valid from a designers
                                          > perspective.However,my comment have been from the
                                          > sales/business perspective. refer the first statement
                                          > in my email
                                          >
                                          > "> From a sales/business perspective:"
                                          >
                                          >
                                          > Hope you see that perspective.
                                          > -Manish
                                          >

                                          Hi Manesh,

                                          Are you talking about a situation where the business/sales goals are at
                                          odds with the goals of the rest of the team?

                                          If that is the case then there is a bigger problem than trying to reduce
                                          the feedback loop for usability testing. Although having stakeholders sit
                                          in on the usability tests, but not as part of the end of sprint demo might
                                          still help to address this.

                                          If there is a misalignment between the goals of the stakeholders and the
                                          implementation team then I would agree that this technique could cause
                                          issues. There would need to be a certain amount of trust and an open
                                          working relationship between the team and the stakeholders for this to
                                          work.

                                          Thanks,

                                          --
                                          Jon Dickinson
                                          Accolade Consulting Ltd.
                                          http://www.accolade-consulting.co.uk
                                          http://www.planitagile.com
                                        Your message has been successfully submitted and would be delivered to recipients shortly.