Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Leverage a small UX team?

Expand Messages
  • Manish Pillewar
    Hi there, Just sharing some thoughts on this ... The very nature of the Agile way of working demands that everyone understand each others domain expertise
    Message 1 of 21 , Jan 31, 2008
    • 0 Attachment
      Hi there,
       
      Just sharing some thoughts on this >
       
      Tim wrote( >>) :



      >>  1) There's clearly a training role, so we're starting to look at which
      >>  skill sets and tasks we should focus the training on. What are the big
      >>  wins for training product owners, developers, etc.?

      The very nature of the Agile way of working demands that everyone understand each others domain expertise that brings max. value to the table. In organisations practising Agile, where designers are absent, the work of user research, IA, etc is usually done by the BA, in addition to the story boards and the narratives that she has to work on. I definitely see the UX designer sharing some load from the BA in that.

      Most developers rack their brains on the information layout and the interactions to use. I think the designer has a lot to contribute there as well. An overall orgnaization thats tuned in to UX design at some level helps. This is exciting because the users on the basis of which the design is proposed will not be limited to Subject Matter Experts( " Yeah, i know how the users use the application. Just ask me whatever doubts you have." scenario) only and may reach out to the real ones through the UX designer.

      >> 2) We plan to operate a core UI team that delivers things like site IA,
      >>  CSS, and Design Patterns for use by the individual teams. What artifacts
      >>  do you think are most important to provide to the individual teams?

      Excellent thought.  I am used to working on ppt for the wireframes and generally keep a stock/ library of the UI elements in ppt format. I dont have to create them all the time. The other artifacts that you can collect are the issue-solution-rationale UI interactions. Do check out http://www.ui-patterns.com . The advantage of arming the devs with this are obvious.

      >>  3) We're thinking that the best times to bring in the UX specialists are
      >>  (a) project inception, prior to forming the team, to help guide initial
      >>  user research (ethnography, persona development, etc.) and (b) during
      >>  release planning (or maybe between each sprint), to help the team ID
      >>  what tools from the world of user-centered design will be most helpful
      >>  for the coming work, and how the UX team can support that.

      This is perfect. You can also choose to involve the designers for POC activities or use their creativity to conceptulize UI solutions during sales pitches or/and do competitive benchmarking exercises as a showcase of Ux capabilities. QA is one more arena where the designer can help in. She can do design reviews or usability evaluations( heuristics,etc), even quick usability testing on the developed screens at every iteration end, to ensure that the UX standards are maintained.

      Would request some folks with more experience to pitch in on this. Hope you find something to take from my unformatted thoughts. I'm figuring out things myself :-)_._,_.___

       

      Cheers!

      Manish Govind Pillewar

      Designer, Thoughtworks, Bangalore, India

      |
      .




      Sent from Yahoo! - a smarter inbox.
    • Adrian Howard
      On 31 Jan 2008, at 09:45, Manish Pillewar wrote: [snip] ... [snip] Not sure that I d describe it as perfect :-) I find that you get a _lot_ more value by
      Message 2 of 21 , Feb 1, 2008
      • 0 Attachment
        On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
        [snip]
        > >> 3) We're thinking that the best times to bring in the UX
        > specialists are
        > >> (a) project inception, prior to forming the team, to help guide
        > initial
        > >> user research (ethnography, persona development, etc.) and (b)
        > during
        > >> release planning (or maybe between each sprint), to help the
        > team ID
        > >> what tools from the world of user-centered design will be most
        > helpful
        > >> for the coming work, and how the UX team can support that.
        >
        > This is perfect. You can also choose to involve the designers for
        > POC activities or use their creativity to conceptulize UI solutions
        > during sales pitches or/and do competitive benchmarking exercises
        > as a showcase of Ux capabilities. QA is one more arena where the
        > designer can help in. She can do design reviews or usability
        > evaluations( heuristics,etc), even quick usability testing on the
        > developed screens at every iteration end, to ensure that the UX
        > standards are maintained.

        [snip]

        Not sure that I'd describe it as perfect :-)

        I find that you get a _lot_ more value by having UX folk involved
        with the team as close to 100% as possible. While there is obviously
        a lot "big" UX work that sits on the customer side, communicating it
        is much better done in person than with documents. There are also
        always UX issues that pop up during development. Having a UX person
        on-hand makes sure that they'll be addressed in an appropriate manner.

        So my perfect set up has at least one UX person as a member of the
        team 100% of the time.

        Which of course sucks as a solution if, like the OP, you only fewer
        UX folk than development teams...

        Assuming that hiring a bunch more UX dudes isn't practical, I'd be
        looking at spending a lot more time in general education of the whole
        team in UX issues. That way there's more chance that UX issues that
        pop up mid-iteration are handled well (even/especially when the
        method is "get a UX dude here stat" :-)

        Cheers,

        Adrian
      • timkieschnick
        Thanks for the good advice. I m clearly not going to get another dozen UX specialists, so I need to leverage my 4 specialists across the people in the
        Message 3 of 21 , Feb 4, 2008
        • 0 Attachment
          Thanks for the good advice. I'm clearly not going to get another
          dozen UX specialists, so I need to leverage my 4 specialists across
          the people in the dedicated teams. Here's what I'm thinking in terms
          of training priorities:

          First, Product Owners:
          - evangelize them on user experience
          - introduce them to the core toolset and the support available
          - think together about how to build UX into the product backlog and
          sprint exit criteria
          - work with them on an ongoing shared product backlog for things
          like creating and implementing design patterns or templates across
          the site

          Second, Business Analysts:
          - evangelize--give them something to live for to replace the
          requirements documents they've been lost in for the last 3 years
          - introduce them to the core toolset and the support available
          - give them the hands-on skills to do things like user-centered
          stories, effective use of personas, and card sorts

          Third, Everybody Else:
          At this point I'll have a skeleton of a support system in place, and
          two people on each team who can tangibly make use of general
          excitement about the user experience.
          - evangelize
          - UX and the product lifecylce
          - when to call for help

          Fourth, Scrummasters:
          - collaborative working session to come up with some shared best
          practices and places to innovate

          Having not done scrum yet myself, I'm probably front-loading too
          much. There will certainly be plenty of informal alignment &
          education going on, but how does this look for the formal training
          part?


          --- In agile-usability@yahoogroups.com, Adrian Howard <adrianh@...>
          wrote:
          >
          > On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
          > [snip]
          > > >> 3) We're thinking that the best times to bring in the UX
          > > specialists are
          > > >> (a) project inception, prior to forming the team, to help
          guide
          > > initial
          > > >> user research (ethnography, persona development, etc.) and
          (b)
          > > during
          > > >> release planning (or maybe between each sprint), to help
          the
          > > team ID
          > > >> what tools from the world of user-centered design will be
          most
          > > helpful
          > > >> for the coming work, and how the UX team can support that.
          > >
          > > This is perfect. You can also choose to involve the designers
          for
          > > POC activities or use their creativity to conceptulize UI
          solutions
          > > during sales pitches or/and do competitive benchmarking
          exercises
          > > as a showcase of Ux capabilities. QA is one more arena where
          the
          > > designer can help in. She can do design reviews or usability
          > > evaluations( heuristics,etc), even quick usability testing on
          the
          > > developed screens at every iteration end, to ensure that the UX
          > > standards are maintained.
          >
          > [snip]
          >
          > Not sure that I'd describe it as perfect :-)
          >
          > I find that you get a _lot_ more value by having UX folk involved
          > with the team as close to 100% as possible. While there is
          obviously
          > a lot "big" UX work that sits on the customer side, communicating
          it
          > is much better done in person than with documents. There are also
          > always UX issues that pop up during development. Having a UX
          person
          > on-hand makes sure that they'll be addressed in an appropriate
          manner.
          >
          > So my perfect set up has at least one UX person as a member of
          the
          > team 100% of the time.
          >
          > Which of course sucks as a solution if, like the OP, you only
          fewer
          > UX folk than development teams...
          >
          > Assuming that hiring a bunch more UX dudes isn't practical, I'd
          be
          > looking at spending a lot more time in general education of the
          whole
          > team in UX issues. That way there's more chance that UX issues
          that
          > pop up mid-iteration are handled well (even/especially when the
          > method is "get a UX dude here stat" :-)
          >
          > Cheers,
          >
          > Adrian
          >
        • Jon Dickinson
          I would be tempted to move Scrum Masters higher on the list. While I think you are right to target product owners first, as they should get the final say on
          Message 4 of 21 , Feb 4, 2008
          • 0 Attachment
            I would be tempted to move Scrum Masters higher on the list. While I think you are right to target product owners first, as they should get the final say on what goes into their product, Scrum Masters should be working closely with the product owners so they have an idea of what is coming up for the next sprint. These people also have the respect of the development team as it is their job to make the teams life easier. It will be useful to build relationships with Scrum Masters early on to figure out how you can fit the UX activities into the sprint cycle.

            I like your idea of building UX into the sprint exit criteria. Do you have any ideas how you are going to achieve this yet?

            Thanks,
            Jon.
            http://www.accolade-consulting.co.uk

            On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:

            Thanks for the good advice. I'm clearly not going to get another
            dozen UX specialists, so I need to leverage my 4 specialists across
            the people in the dedicated teams. Here's what I'm thinking in terms
            of training priorities:

            First, Product Owners:
            - evangelize them on user experience
            - introduce them to the core toolset and the support available
            - think together about how to build UX into the product backlog and
            sprint exit criteria
            - work with them on an ongoing shared product backlog for things
            like creating and implementing design patterns or templates across
            the site

            Second, Business Analysts:
            - evangelize--give them something to live for to replace the
            requirements documents they've been lost in for the last 3 years
            - introduce them to the core toolset and the support available
            - give them the hands-on skills to do things like user-centered
            stories, effective use of personas, and card sorts

            Third, Everybody Else:
            At this point I'll have a skeleton of a support system in place, and
            two people on each team who can tangibly make use of general
            excitement about the user experience.
            - evangelize
            - UX and the product lifecylce
            - when to call for help

            Fourth, Scrummasters:
            - collaborative working session to come up with some shared best
            practices and places to innovate

            Having not done scrum yet myself, I'm probably front-loading too
            much. There will certainly be plenty of informal alignment &
            education going on, but how does this look for the formal training
            part?

            --- In agile-usability@yahoogroups.com, Adrian Howard <adrianh@...>


            wrote:
            >
            > On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
            > [snip]
            > > >> 3) We're thinking that the best times to bring in the UX
            > > specialists are
            > > >> (a) project inception, prior to forming the team, to help
            guide
            > > initial
            > > >> user research (ethnography, persona development, etc.) and
            (b)
            > > during
            > > >> release planning (or maybe between each sprint), to help
            the
            > > team ID
            > > >> what tools from the world of user-centered design will be
            most
            > > helpful
            > > >> for the coming work, and how the UX team can support that.
            > >
            > > This is perfect. You can also choose to involve the designers
            for
            > > POC activities or use their creativity to conceptulize UI
            solutions
            > > during sales pitches or/and do competitive benchmarking
            exercises
            > > as a showcase of Ux capabilities. QA is one more arena where
            the
            > > designer can help in. She can do design reviews or usability
            > > evaluations( heuristics,etc), even quick usability testing on
            the
            > > developed screens at every iteration end, to ensure that the UX
            > > standards are maintained.
            >
            > [snip]
            >
            > Not sure that I'd describe it as perfect :-)
            >
            > I find that you get a _lot_ more value by having UX folk involved
            > with the team as close to 100% as possible. While there is
            obviously
            > a lot "big" UX work that sits on the customer side, communicating
            it
            > is much better done in person than with documents. There are also
            > always UX issues that pop up during development. Having a UX
            person
            > on-hand makes sure that they'll be addressed in an appropriate
            manner.
            >
            > So my perfect set up has at least one UX person as a member of
            the
            > team 100% of the time.
            >
            > Which of course sucks as a solution if, like the OP, you only
            fewer
            > UX folk than development teams...
            >
            > Assuming that hiring a bunch more UX dudes isn't practical, I'd
            be
            > looking at spending a lot more time in general education of the
            whole
            > team in UX issues. That way there's more chance that UX issues
            that
            > pop up mid-iteration are handled well (even/especially when the
            > method is "get a UX dude here stat" :-)
            >
            > Cheers,
            >
            > Adrian
            >


          • timkieschnick
            ... you have ... My ideas so far: 1) Be sure exit criteria is framed from the user s perspective, e.g. Katherine (our primary person) can cancel an order.
            Message 5 of 21 , Feb 4, 2008
            • 0 Attachment
              --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
              wrote:
              > I like your idea of building UX into the sprint exit criteria. Do
              you have
              > any ideas how you are going to achieve this yet?

              My ideas so far:

              1) Be sure exit criteria is framed from the user's perspective,
              e.g. "Katherine (our primary person) can cancel an order." rather
              than "implement cancel order functionality"

              2) I think there may be some instances where it will make sense to
              put a usability metric on the exit criteria, e.g., "Users can cancel
              an order, and they rate their experience an average of at least 4 on
              either the SUS or the question, "Compared to other sites, how easy
              was this to use?"

              This metric would be supported by what I'm calling "commoditized
              user research"--this is something that we've started doing that sort
              of turns most user research into a commodity--it makes it very
              simple for the project team to get user feedback. I can say much
              more on that if anyone's interested.
            • eric.idebro
              ... I can say much more on that if anyone s interested. ... Please do!
              Message 6 of 21 , Feb 4, 2008
              • 0 Attachment
                >
                > My ideas so far:
                I can say much more on that if anyone's interested.

                >







                Please do!
              • Jon Dickinson
                ... Have you considered a team based approach to training rather than role based?
                Message 7 of 21 , Feb 5, 2008
                • 0 Attachment
                  On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:


                  Having not done scrum yet myself, I'm probably front-loading too
                  much. There will certainly be plenty of informal alignment &
                  education going on, but how does this look for the formal training
                  part?












                  Have you considered a team based approach to training rather than role based?
                • Jon Dickinson
                  Would you run the usability test as part of the demo of the product increment? I like the idea of the demo being the usability test and the stakeholders being
                  Message 8 of 21 , Feb 5, 2008
                  • 0 Attachment
                    Would you run the usability test as part of the demo of the product increment? I like the idea of the demo being the usability test and the stakeholders being observers of the usability test.

                    > This metric would be supported by what I'm calling "commoditized
                    > user research"

                    Sounds like a blog post in the making. I would write this up outside of the list and then send a link to the list. I'm sure it would be a great way to get feedback. Not that I'm a list administrator, I've just seen people do this in the XP list and it works very well.

                    Jon.




                    On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:

                    --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                    wrote:
                    > I like your idea of building UX into the sprint exit criteria. Do
                    you have
                    > any ideas how you are going to achieve this yet?

                    My ideas so far:

                    1) Be sure exit criteria is framed from the user's perspective,
                    e.g. "Katherine (our primary person) can cancel an order." rather
                    than "implement cancel order functionality"

                    2) I think there may be some instances where it will make sense to
                    put a usability metric on the exit criteria, e.g., "Users can cancel
                    an order, and they rate their experience an average of at least 4 on
                    either the SUS or the question, "Compared to other sites, how easy
                    was this to use?"

                    This metric would be supported by what I'm calling "commoditized
                    user research"--this is something that we've started doing that sort
                    of turns most user research into a commodity--it makes it very
                    simple for the project team to get user feedback. I can say much
                    more on that if anyone's interested.


                  • Adrian Howard
                    ... That was what I was just about to suggest. Once you have an organisation that accepts that UX issues are important (and if they re financing a UX
                    Message 9 of 21 , Feb 5, 2008
                    • 0 Attachment
                      On 5 Feb 2008, at 09:04, Jon Dickinson wrote:

                      >> On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:
                      >>
                      >>>
                      >>> Having not done scrum yet myself, I'm probably front-loading too
                      >>> much. There will certainly be plenty of informal alignment &
                      >>> education going on, but how does this look for the formal training
                      >>> part?
                      >
                      > Have you considered a team based approach to training rather than role
                      > based?

                      That was what I was just about to suggest. Once you have an
                      organisation that accepts that UX issues are important (and if
                      they're financing a UX department than my guess is the answer to that
                      is yes :-) then I'd focus on educating project teams as a whole
                      rather than roles.

                      In my experience UX in agile teams is as much bottom-up as it is top-
                      down (like most other things in agile teams). I think you'll see see
                      more progress more quickly if you educate a team at a time, rather
                      than a layer that cuts across all teams.

                      Cheers,

                      Adrian
                    • Adrian Howard
                      ... [snip] I ve only managed to do it a couple of times but I find whole-team usability tests very useful. While he s not talking about an agile environment
                      Message 10 of 21 , Feb 5, 2008
                      • 0 Attachment
                        On 5 Feb 2008, at 09:53, Jon Dickinson wrote:

                        > Would you run the usability test as part of the demo of the product
                        > increment? I like the idea of the demo being the usability test and
                        > the stakeholders being observers of the usability test.
                        [snip]

                        I've only managed to do it a couple of times but I find whole-team
                        usability tests very useful. While he's not talking about an agile
                        environment Jared Spool has a nice bit on testing with large groups
                        in these two blog posts:

                        * http://www.uie.com/brainsparks/2007/09/13/usability-tests-with-30-
                        observers/
                        * http://www.uie.com/brainsparks/2007/09/18/more-on-usability-tests-
                        with-30-observers/

                        Cheers,

                        Adrian
                      • Adrian Howard
                        On 4 Feb 2008, at 22:59, timkieschnick wrote: [snip] ... [snip] While I m not a huge metric fan, if you re having problem convincing folk about the importance
                        Message 11 of 21 , Feb 5, 2008
                        • 0 Attachment
                          On 4 Feb 2008, at 22:59, timkieschnick wrote:
                          [snip]
                          > 2) I think there may be some instances where it will make sense to
                          > put a usability metric on the exit criteria, e.g., "Users can cancel
                          > an order, and they rate their experience an average of at least 4 on
                          > either the SUS or the question, "Compared to other sites, how easy
                          > was this to use?"
                          [snip]

                          While I'm not a huge metric fan, if you're having problem convincing
                          folk about the importance of UX issues having a graph of SUS numbers
                          over time in the wall along with the burndown chart, et al, can be
                          useful :-)

                          Cheers,

                          Adrian
                        • Gart, Mitchell
                          In our company we have a few UX people spread over several development teams. Each UX person has to split their time between a few development projects. Each
                          Message 12 of 21 , Feb 5, 2008
                          • 0 Attachment
                            In our company we have a few UX people spread over several development
                            teams. Each UX person has to split their time between a few development
                            projects. Each team has several full time developers and one part time
                            UX person.

                            I find that the most important thing is having a good, trusting working
                            relationship. If the developers trust the UX person, and vice versa,
                            they will be able to work together with this kind of full time - part
                            time division of labor. If they don't have a trusting relationship,
                            things will not go so well.

                            - Mitch Gart
                          • timkieschnick
                            ... and the ... Has anyone done what Jon s suggesting? Somehow combinging usability test with the demo? I like the idea of showing live user testing to the
                            Message 13 of 21 , Mar 3, 2008
                            • 0 Attachment
                              --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                              wrote:
                              >
                              > Would you run the usability test as part of the demo of the product
                              > increment? I like the idea of the demo being the usability test
                              and the
                              > stakeholders being observers of the usability test.
                              >

                              Has anyone done what Jon's suggesting? Somehow combinging usability
                              test with the demo?

                              I like the idea of showing live user testing to the stakeholders,
                              but it sounds problematic in terms of being able to show a
                              representative sample of users to the stakeholders--I wouldn't want
                              the stakeholders to see the one outlyer test, and I don't know how
                              I'd show them 4-5 users in one demo.

                              The only way I can think of to show them multiple users would be to
                              splice together video clips of representative user tests. This would
                              be fantastic in a demo, but too much work to do regularly.

                              Anyone have experience with this?
                            • Manish Pillewar
                              From a sales/business perspective: Q. Would you run the usability test as part of the demo of the product increment? Comment: It has its risks. The users
                              Message 14 of 21 , Mar 3, 2008
                              • 0 Attachment
                                From a sales/business perspective:

                                Q. Would you run the usability test as part of the
                                demo of the product increment?

                                Comment: It has its risks. The users nominated for
                                testing need to be very close to the persona's you
                                created during the application design. Provided you
                                have done all the risks mitigation, its should help
                                drive the point home asap.


                                Q. I like the idea of the demo being the usability
                                test
                                and the stakeholders being observers of the usability
                                test.

                                Comment: The challenge here is to moderate the test
                                show while you bring out the best features of the
                                product. Live usability testing may bring in
                                additional viewpoints on how the client organization
                                works and how certain additional features would have
                                been great to have. This could potentially become a
                                point of negotiation on the cost estimates for your
                                product.


                                Q. Has anyone done what Jon's suggesting? Somehow
                                combining usability test with the demo? I like the
                                idea of showing live user testing to the stakeholders,
                                but it sounds problematic in terms of being able to
                                show a representative sample of users to the
                                stakeholders- -I wouldn't want the stakeholders to see
                                the one outlyer test, and I don't know how I'd show
                                them 4-5 users in one demo. The only way I can think
                                of to show them multiple users would be to splice
                                together video clips of representative user tests.
                                This would be fantastic in a demo, but too much work
                                to do regularly.Anyone have experience with this?

                                Comment: Showing a short demo video coupled with the
                                statistics generated from the usability test seems to
                                be a better idea, in my experience. It not only shows
                                that you have taken great care while designing the
                                product,etc. but also generates less queries and
                                lesser debates, as the data is for all to see.

                                I've kept aside design ethics which would otherwise
                                dictate that in case the user testing fails, its the
                                feedback thats important, and you want to sell usable
                                software only, even if it means that you don't make a
                                sale at all.





                                Thanks and Regards
                                Manish Govind Pillewar
                                Sr. User Experience Designer
                                Thoughtworks India Pvt. Ltd.Bangalore-India

                                Tel. +91 9880566951 (M)
                                +91 80 41113967 (Eve.)
                                Smith & Wesson: The original point and click interface :-)



                                ___________________________________________________________
                                Rise to the challenge for Sport Relief with Yahoo! For Good

                                http://uk.promotions.yahoo.com/forgood/
                              • Jeff Lopez-Stuit
                                I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo
                                Message 15 of 21 , Mar 3, 2008
                                • 0 Attachment
                                  I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo the testing itself.   The objective of the demo should be to show what value was produced, not how the work was done.  You can provide a quick summary the tests, but the focus should be on the specific measurable improvements in usability that were identified and addressed during the sprint.

                                  Take care...

                                  Jeff Lopez-Stuit, CSM, CSPO
                                  SolutionsIQ

                                  On Mon, Mar 3, 2008 at 3:14 PM, timkieschnick <tim.kieschnick@...> wrote:

                                  --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                                  wrote:
                                  >
                                  > Would you run the usability test as part of the demo of the product
                                  > increment? I like the idea of the demo being the usability test
                                  and the
                                  > stakeholders being observers of the usability test.
                                  >


                                  .


                                • Jon Dickinson
                                  On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit ... I am making the assumption that the product owners already understand the value of usability testing,
                                  Message 16 of 21 , Mar 8, 2008
                                  • 0 Attachment
                                    On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit
                                    <jeff.lopezstuit@...> wrote:

                                    > I think it would be much more meaningful to the product owners if you
                                    > could
                                    > demonstrate the *value* of the testing you did during the sprint, rather
                                    > than demo the testing itself.

                                    I am making the assumption that the product owners already understand the
                                    value of usability testing, so it is not necessary to demonstrate the
                                    value of the technique.

                                    > The objective of the demo should be to show
                                    > what value was produced, not how the work was done.

                                    Agreed. The product managers attend a usability test not to see how a
                                    usability test is performed but to shorten the feedback loop from the
                                    results of the test.

                                    > You can provide a quick
                                    > summary the tests, but the focus should be on the specific measurable
                                    > improvements in usability that were identified and addressed during the
                                    > sprint.


                                    Over the course of a few end of sprint usability tests the product
                                    managers should get a good first hand impression of whether usability
                                    improvements are being made during the sprints. This doesn't stop one
                                    keeping records of the tests and producing measurements to show usability
                                    improvements over time, if this provides any additional value.

                                    --
                                    Jon Dickinson
                                    Accolade Consulting
                                    http://www.accolade-consulting.co.uk
                                  • Jon Dickinson
                                    On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar ... Is this not a useful exercise to validate your personas are correct? ... Assuming your product has more
                                    Message 17 of 21 , Mar 8, 2008
                                    • 0 Attachment
                                      On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
                                      <manish1022@...> wrote:

                                      >
                                      >
                                      >
                                      > From a sales/business perspective:
                                      >
                                      > Q. Would you run the usability test as part of the
                                      > demo of the product increment?
                                      >
                                      > Comment: It has its risks. The users nominated for
                                      > testing need to be very close to the persona's you
                                      > created during the application design.

                                      Is this not a useful exercise to validate your personas are correct?

                                      >
                                      > Q. I like the idea of the demo being the usability
                                      > test
                                      > and the stakeholders being observers of the usability
                                      > test.
                                      >
                                      > Comment: The challenge here is to moderate the test
                                      > show while you bring out the best features of the
                                      > product. Live usability testing may bring in
                                      > additional viewpoints on how the client organization
                                      > works and how certain additional features would have
                                      > been great to have. This could potentially become a
                                      > point of negotiation on the cost estimates for your
                                      > product.

                                      Assuming your product has more than one sprint, and therefore more than
                                      one usability test, before releasing why would this be a bad thing?

                                      > Comment: Showing a short demo video coupled with the
                                      > statistics generated from the usability test seems to
                                      > be a better idea, in my experience. It not only shows
                                      > that you have taken great care while designing the
                                      > product,etc. but also generates less queries and
                                      > lesser debates, as the data is for all to see.

                                      Why would you want to reduce debate? The idea of running a usibility test
                                      at the end of the sprint is generate discussion to help shape the work for
                                      the next sprint.

                                      --
                                      Jon Dickinson
                                      Accolade Consulting
                                      http://www.accolade-consulting.co.uk
                                    • Jon Dickinson
                                      On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick ... I can see that it would be a risk if the results of one bad test/demonstration to
                                      Message 18 of 21 , Mar 8, 2008
                                      • 0 Attachment
                                        On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick <tim.kieschnick@...>
                                        wrote:

                                        > I like the idea of showing live user testing to the stakeholders,
                                        > but it sounds problematic in terms of being able to show a
                                        > representative sample of users to the stakeholders--I wouldn't want
                                        > the stakeholders to see the one outlyer test, and I don't know how
                                        > I'd show them 4-5 users in one demo.

                                        I can see that it would be a risk if the results of one bad
                                        test/demonstration to the stakeholders sends the team off in the wrong
                                        direction for an entire sprint.

                                        From reading your excellent experience report:
                                        http://timiti.blogspot.com/2008/02/user-research-as-commodity.html, the
                                        following passage strikes me as providing two possible solutions:

                                        "Every other Thursday morning, four or five brokers (our target audience)
                                        would show up at our offices in Pasadena. Our user research specialist
                                        would work with each participant for about an hour. While this testing
                                        took place in Pasadena, we piped audio and video of the testing up north
                                        to Pleasanton so the team could watch in real-time and IM questions and
                                        comments to the moderator in Pasadena."

                                        1) Persuade the stakeholders to spend the entire morning at the end of the
                                        sprint observing the usability tests. This is my ideal.
                                        2) Have the stakeholders sit in as observers on the test with the last
                                        participant. If the results of this test turn out to be radically
                                        different to the earlier tests, you have the results of the previous tests
                                        to demonstrate that this was the exception.

                                        If the stakeholders are limited to observing a single test, it is probably
                                        worth giving them a half hour demo of the work completed before the
                                        usability test so they have an initial idea of what work has been
                                        completed. You don't want one bad test to negate all the hard work the
                                        team has performed during the sprint.

                                        Cheers,

                                        --
                                        Jon Dickinson
                                        Accolade Consulting
                                        http://www.accolade-consulting.co.uk
                                      • Manish Pillewar
                                        Hi Jon, Your questions are of course valid from a designers perspective.However,my comment have been from the sales/business perspective. refer the first
                                        Message 19 of 21 , Mar 9, 2008
                                        • 0 Attachment
                                          Hi Jon,
                                          Your questions are of course valid from a designers
                                          perspective.However,my comment have been from the
                                          sales/business perspective. refer the first statement
                                          in my email

                                          "> From a sales/business perspective:"


                                          Hope you see that perspective.
                                          -Manish


                                          On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
                                          <manish1022@...> wrote:

                                          >
                                          >
                                          >
                                          > From a sales/business perspective:
                                          >
                                          > Q. Would you run the usability test as part of the
                                          > demo of the product increment?
                                          >
                                          > Comment: It has its risks. The users nominated for
                                          > testing need to be very close to the persona's you
                                          > created during the application design.

                                          Is this not a useful exercise to validate your
                                          personas are correct?

                                          >
                                          > Q. I like the idea of the demo being the usability
                                          > test
                                          > and the stakeholders being observers of the
                                          usability
                                          > test.
                                          >
                                          > Comment: The challenge here is to moderate the test
                                          > show while you bring out the best features of the
                                          > product. Live usability testing may bring in
                                          > additional viewpoints on how the client organization
                                          > works and how certain additional features would have
                                          > been great to have. This could potentially become a
                                          > point of negotiation on the cost estimates for your
                                          > product.

                                          Assuming your product has more than one sprint, and
                                          therefore more than

                                          one usability test, before releasing why would this be
                                          a bad thing?

                                          > Comment: Showing a short demo video coupled with the
                                          > statistics generated from the usability test seems
                                          to
                                          > be a better idea, in my experience. It not only
                                          shows
                                          > that you have taken great care while designing the
                                          > product,etc. but also generates less queries and
                                          > lesser debates, as the data is for all to see.

                                          Why would you want to reduce debate? The idea of
                                          running a usibility
                                          test
                                          at the end of the sprint is generate discussion to
                                          help shape the work
                                          for
                                          the next sprint.

                                          --
                                          Jon Dickinson
                                          Accolade Consulting
                                          http://www.accolade-consulting.co.uk

                                          Thanks and Regards
                                          Manish Govind Pillewar
                                          Sr. User Experience Designer
                                          Thoughtworks India Pvt. Ltd.Bangalore-India

                                          Tel. +91 9880566951 (M)
                                          +91 80 41113967 (Eve.)
                                          Smith & Wesson: The original point and click interface :-)



                                          __________________________________________________________
                                          Sent from Yahoo! Mail.
                                          The World's Favourite Email http://uk.docs.yahoo.com/nowyoucan.html
                                        • Jon Dickinson
                                          On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar ... Hi Manesh, Are you talking about a situation where the business/sales goals are at odds with the goals
                                          Message 20 of 21 , Mar 10, 2008
                                          • 0 Attachment
                                            On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar
                                            <manish1022@...> wrote:

                                            > Hi Jon,
                                            > Your questions are of course valid from a designers
                                            > perspective.However,my comment have been from the
                                            > sales/business perspective. refer the first statement
                                            > in my email
                                            >
                                            > "> From a sales/business perspective:"
                                            >
                                            >
                                            > Hope you see that perspective.
                                            > -Manish
                                            >

                                            Hi Manesh,

                                            Are you talking about a situation where the business/sales goals are at
                                            odds with the goals of the rest of the team?

                                            If that is the case then there is a bigger problem than trying to reduce
                                            the feedback loop for usability testing. Although having stakeholders sit
                                            in on the usability tests, but not as part of the end of sprint demo might
                                            still help to address this.

                                            If there is a misalignment between the goals of the stakeholders and the
                                            implementation team then I would agree that this technique could cause
                                            issues. There would need to be a certain amount of trust and an open
                                            working relationship between the team and the stakeholders for this to
                                            work.

                                            Thanks,

                                            --
                                            Jon Dickinson
                                            Accolade Consulting Ltd.
                                            http://www.accolade-consulting.co.uk
                                            http://www.planitagile.com
                                          Your message has been successfully submitted and would be delivered to recipients shortly.