Loading ...
Sorry, an error occurred while loading the content.

Leverage a small UX team?

Expand Messages
  • timkieschnick
    I have 2 UX professionals (IA/User Research/Interaction Design, etc.), and I m hoping to hire two more. Our Web shop is starting Scrum. We ve launched 4 scrum
    Message 1 of 21 , Jan 30, 2008
    • 0 Attachment
      I have 2 UX professionals (IA/User Research/Interaction Design, etc.),
      and I'm hoping to hire two more.

      Our Web shop is starting Scrum. We've launched 4 scrum teams so far, and
      I expect we'll build up to maybe 20 teams over the next year or two as
      we convert staff & projects from the traditional methodologies to scrum.

      We're trying to make a clear distinction between chickens and pigs, and
      the teams are being set up to have team members fully committed to the
      team, rather than floating across teams.

      In this environment, what advice do you have for how to best use my
      small pool of user experience specialists? Here's my initial thinking:

      1) There's clearly a training role, so we're starting to look at which
      skill sets and tasks we should focus the training on. What are the big
      wins for training product owners, developers, etc.?

      2) We plan to operate a core UI team that delivers things like site IA,
      CSS, and Design Patterns for use by the individual teams. What artifacts
      do you think are most important to provide to the individual teams?

      3) We're thinking that the best times to bring in the UX specialists are
      (a) project inception, prior to forming the team, to help guide initial
      user research (ethnography, persona development, etc.) and (b) during
      release planning (or maybe between each sprint), to help the team ID
      what tools from the world of user-centered design will be most helpful
      for the coming work, and how the UX team can support that.

      Overall, what's your experience and advice on when and how to use the
      specialists across a much larger community?
    • Manish Pillewar
      Hi there, Just sharing some thoughts on this ... The very nature of the Agile way of working demands that everyone understand each others domain expertise
      Message 2 of 21 , Jan 31, 2008
      • 0 Attachment
        Hi there,
         
        Just sharing some thoughts on this >
         
        Tim wrote( >>) :



        >>  1) There's clearly a training role, so we're starting to look at which
        >>  skill sets and tasks we should focus the training on. What are the big
        >>  wins for training product owners, developers, etc.?

        The very nature of the Agile way of working demands that everyone understand each others domain expertise that brings max. value to the table. In organisations practising Agile, where designers are absent, the work of user research, IA, etc is usually done by the BA, in addition to the story boards and the narratives that she has to work on. I definitely see the UX designer sharing some load from the BA in that.

        Most developers rack their brains on the information layout and the interactions to use. I think the designer has a lot to contribute there as well. An overall orgnaization thats tuned in to UX design at some level helps. This is exciting because the users on the basis of which the design is proposed will not be limited to Subject Matter Experts( " Yeah, i know how the users use the application. Just ask me whatever doubts you have." scenario) only and may reach out to the real ones through the UX designer.

        >> 2) We plan to operate a core UI team that delivers things like site IA,
        >>  CSS, and Design Patterns for use by the individual teams. What artifacts
        >>  do you think are most important to provide to the individual teams?

        Excellent thought.  I am used to working on ppt for the wireframes and generally keep a stock/ library of the UI elements in ppt format. I dont have to create them all the time. The other artifacts that you can collect are the issue-solution-rationale UI interactions. Do check out http://www.ui-patterns.com . The advantage of arming the devs with this are obvious.

        >>  3) We're thinking that the best times to bring in the UX specialists are
        >>  (a) project inception, prior to forming the team, to help guide initial
        >>  user research (ethnography, persona development, etc.) and (b) during
        >>  release planning (or maybe between each sprint), to help the team ID
        >>  what tools from the world of user-centered design will be most helpful
        >>  for the coming work, and how the UX team can support that.

        This is perfect. You can also choose to involve the designers for POC activities or use their creativity to conceptulize UI solutions during sales pitches or/and do competitive benchmarking exercises as a showcase of Ux capabilities. QA is one more arena where the designer can help in. She can do design reviews or usability evaluations( heuristics,etc), even quick usability testing on the developed screens at every iteration end, to ensure that the UX standards are maintained.

        Would request some folks with more experience to pitch in on this. Hope you find something to take from my unformatted thoughts. I'm figuring out things myself :-)_._,_.___

         

        Cheers!

        Manish Govind Pillewar

        Designer, Thoughtworks, Bangalore, India

        |
        .




        Sent from Yahoo! - a smarter inbox.
      • Adrian Howard
        On 31 Jan 2008, at 09:45, Manish Pillewar wrote: [snip] ... [snip] Not sure that I d describe it as perfect :-) I find that you get a _lot_ more value by
        Message 3 of 21 , Feb 1, 2008
        • 0 Attachment
          On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
          [snip]
          > >> 3) We're thinking that the best times to bring in the UX
          > specialists are
          > >> (a) project inception, prior to forming the team, to help guide
          > initial
          > >> user research (ethnography, persona development, etc.) and (b)
          > during
          > >> release planning (or maybe between each sprint), to help the
          > team ID
          > >> what tools from the world of user-centered design will be most
          > helpful
          > >> for the coming work, and how the UX team can support that.
          >
          > This is perfect. You can also choose to involve the designers for
          > POC activities or use their creativity to conceptulize UI solutions
          > during sales pitches or/and do competitive benchmarking exercises
          > as a showcase of Ux capabilities. QA is one more arena where the
          > designer can help in. She can do design reviews or usability
          > evaluations( heuristics,etc), even quick usability testing on the
          > developed screens at every iteration end, to ensure that the UX
          > standards are maintained.

          [snip]

          Not sure that I'd describe it as perfect :-)

          I find that you get a _lot_ more value by having UX folk involved
          with the team as close to 100% as possible. While there is obviously
          a lot "big" UX work that sits on the customer side, communicating it
          is much better done in person than with documents. There are also
          always UX issues that pop up during development. Having a UX person
          on-hand makes sure that they'll be addressed in an appropriate manner.

          So my perfect set up has at least one UX person as a member of the
          team 100% of the time.

          Which of course sucks as a solution if, like the OP, you only fewer
          UX folk than development teams...

          Assuming that hiring a bunch more UX dudes isn't practical, I'd be
          looking at spending a lot more time in general education of the whole
          team in UX issues. That way there's more chance that UX issues that
          pop up mid-iteration are handled well (even/especially when the
          method is "get a UX dude here stat" :-)

          Cheers,

          Adrian
        • timkieschnick
          Thanks for the good advice. I m clearly not going to get another dozen UX specialists, so I need to leverage my 4 specialists across the people in the
          Message 4 of 21 , Feb 4, 2008
          • 0 Attachment
            Thanks for the good advice. I'm clearly not going to get another
            dozen UX specialists, so I need to leverage my 4 specialists across
            the people in the dedicated teams. Here's what I'm thinking in terms
            of training priorities:

            First, Product Owners:
            - evangelize them on user experience
            - introduce them to the core toolset and the support available
            - think together about how to build UX into the product backlog and
            sprint exit criteria
            - work with them on an ongoing shared product backlog for things
            like creating and implementing design patterns or templates across
            the site

            Second, Business Analysts:
            - evangelize--give them something to live for to replace the
            requirements documents they've been lost in for the last 3 years
            - introduce them to the core toolset and the support available
            - give them the hands-on skills to do things like user-centered
            stories, effective use of personas, and card sorts

            Third, Everybody Else:
            At this point I'll have a skeleton of a support system in place, and
            two people on each team who can tangibly make use of general
            excitement about the user experience.
            - evangelize
            - UX and the product lifecylce
            - when to call for help

            Fourth, Scrummasters:
            - collaborative working session to come up with some shared best
            practices and places to innovate

            Having not done scrum yet myself, I'm probably front-loading too
            much. There will certainly be plenty of informal alignment &
            education going on, but how does this look for the formal training
            part?


            --- In agile-usability@yahoogroups.com, Adrian Howard <adrianh@...>
            wrote:
            >
            > On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
            > [snip]
            > > >> 3) We're thinking that the best times to bring in the UX
            > > specialists are
            > > >> (a) project inception, prior to forming the team, to help
            guide
            > > initial
            > > >> user research (ethnography, persona development, etc.) and
            (b)
            > > during
            > > >> release planning (or maybe between each sprint), to help
            the
            > > team ID
            > > >> what tools from the world of user-centered design will be
            most
            > > helpful
            > > >> for the coming work, and how the UX team can support that.
            > >
            > > This is perfect. You can also choose to involve the designers
            for
            > > POC activities or use their creativity to conceptulize UI
            solutions
            > > during sales pitches or/and do competitive benchmarking
            exercises
            > > as a showcase of Ux capabilities. QA is one more arena where
            the
            > > designer can help in. She can do design reviews or usability
            > > evaluations( heuristics,etc), even quick usability testing on
            the
            > > developed screens at every iteration end, to ensure that the UX
            > > standards are maintained.
            >
            > [snip]
            >
            > Not sure that I'd describe it as perfect :-)
            >
            > I find that you get a _lot_ more value by having UX folk involved
            > with the team as close to 100% as possible. While there is
            obviously
            > a lot "big" UX work that sits on the customer side, communicating
            it
            > is much better done in person than with documents. There are also
            > always UX issues that pop up during development. Having a UX
            person
            > on-hand makes sure that they'll be addressed in an appropriate
            manner.
            >
            > So my perfect set up has at least one UX person as a member of
            the
            > team 100% of the time.
            >
            > Which of course sucks as a solution if, like the OP, you only
            fewer
            > UX folk than development teams...
            >
            > Assuming that hiring a bunch more UX dudes isn't practical, I'd
            be
            > looking at spending a lot more time in general education of the
            whole
            > team in UX issues. That way there's more chance that UX issues
            that
            > pop up mid-iteration are handled well (even/especially when the
            > method is "get a UX dude here stat" :-)
            >
            > Cheers,
            >
            > Adrian
            >
          • Jon Dickinson
            I would be tempted to move Scrum Masters higher on the list. While I think you are right to target product owners first, as they should get the final say on
            Message 5 of 21 , Feb 4, 2008
            • 0 Attachment
              I would be tempted to move Scrum Masters higher on the list. While I think you are right to target product owners first, as they should get the final say on what goes into their product, Scrum Masters should be working closely with the product owners so they have an idea of what is coming up for the next sprint. These people also have the respect of the development team as it is their job to make the teams life easier. It will be useful to build relationships with Scrum Masters early on to figure out how you can fit the UX activities into the sprint cycle.

              I like your idea of building UX into the sprint exit criteria. Do you have any ideas how you are going to achieve this yet?

              Thanks,
              Jon.
              http://www.accolade-consulting.co.uk

              On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:

              Thanks for the good advice. I'm clearly not going to get another
              dozen UX specialists, so I need to leverage my 4 specialists across
              the people in the dedicated teams. Here's what I'm thinking in terms
              of training priorities:

              First, Product Owners:
              - evangelize them on user experience
              - introduce them to the core toolset and the support available
              - think together about how to build UX into the product backlog and
              sprint exit criteria
              - work with them on an ongoing shared product backlog for things
              like creating and implementing design patterns or templates across
              the site

              Second, Business Analysts:
              - evangelize--give them something to live for to replace the
              requirements documents they've been lost in for the last 3 years
              - introduce them to the core toolset and the support available
              - give them the hands-on skills to do things like user-centered
              stories, effective use of personas, and card sorts

              Third, Everybody Else:
              At this point I'll have a skeleton of a support system in place, and
              two people on each team who can tangibly make use of general
              excitement about the user experience.
              - evangelize
              - UX and the product lifecylce
              - when to call for help

              Fourth, Scrummasters:
              - collaborative working session to come up with some shared best
              practices and places to innovate

              Having not done scrum yet myself, I'm probably front-loading too
              much. There will certainly be plenty of informal alignment &
              education going on, but how does this look for the formal training
              part?

              --- In agile-usability@yahoogroups.com, Adrian Howard <adrianh@...>


              wrote:
              >
              > On 31 Jan 2008, at 09:45, Manish Pillewar wrote:
              > [snip]
              > > >> 3) We're thinking that the best times to bring in the UX
              > > specialists are
              > > >> (a) project inception, prior to forming the team, to help
              guide
              > > initial
              > > >> user research (ethnography, persona development, etc.) and
              (b)
              > > during
              > > >> release planning (or maybe between each sprint), to help
              the
              > > team ID
              > > >> what tools from the world of user-centered design will be
              most
              > > helpful
              > > >> for the coming work, and how the UX team can support that.
              > >
              > > This is perfect. You can also choose to involve the designers
              for
              > > POC activities or use their creativity to conceptulize UI
              solutions
              > > during sales pitches or/and do competitive benchmarking
              exercises
              > > as a showcase of Ux capabilities. QA is one more arena where
              the
              > > designer can help in. She can do design reviews or usability
              > > evaluations( heuristics,etc), even quick usability testing on
              the
              > > developed screens at every iteration end, to ensure that the UX
              > > standards are maintained.
              >
              > [snip]
              >
              > Not sure that I'd describe it as perfect :-)
              >
              > I find that you get a _lot_ more value by having UX folk involved
              > with the team as close to 100% as possible. While there is
              obviously
              > a lot "big" UX work that sits on the customer side, communicating
              it
              > is much better done in person than with documents. There are also
              > always UX issues that pop up during development. Having a UX
              person
              > on-hand makes sure that they'll be addressed in an appropriate
              manner.
              >
              > So my perfect set up has at least one UX person as a member of
              the
              > team 100% of the time.
              >
              > Which of course sucks as a solution if, like the OP, you only
              fewer
              > UX folk than development teams...
              >
              > Assuming that hiring a bunch more UX dudes isn't practical, I'd
              be
              > looking at spending a lot more time in general education of the
              whole
              > team in UX issues. That way there's more chance that UX issues
              that
              > pop up mid-iteration are handled well (even/especially when the
              > method is "get a UX dude here stat" :-)
              >
              > Cheers,
              >
              > Adrian
              >


            • timkieschnick
              ... you have ... My ideas so far: 1) Be sure exit criteria is framed from the user s perspective, e.g. Katherine (our primary person) can cancel an order.
              Message 6 of 21 , Feb 4, 2008
              • 0 Attachment
                --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                wrote:
                > I like your idea of building UX into the sprint exit criteria. Do
                you have
                > any ideas how you are going to achieve this yet?

                My ideas so far:

                1) Be sure exit criteria is framed from the user's perspective,
                e.g. "Katherine (our primary person) can cancel an order." rather
                than "implement cancel order functionality"

                2) I think there may be some instances where it will make sense to
                put a usability metric on the exit criteria, e.g., "Users can cancel
                an order, and they rate their experience an average of at least 4 on
                either the SUS or the question, "Compared to other sites, how easy
                was this to use?"

                This metric would be supported by what I'm calling "commoditized
                user research"--this is something that we've started doing that sort
                of turns most user research into a commodity--it makes it very
                simple for the project team to get user feedback. I can say much
                more on that if anyone's interested.
              • eric.idebro
                ... I can say much more on that if anyone s interested. ... Please do!
                Message 7 of 21 , Feb 4, 2008
                • 0 Attachment
                  >
                  > My ideas so far:
                  I can say much more on that if anyone's interested.

                  >







                  Please do!
                • Jon Dickinson
                  ... Have you considered a team based approach to training rather than role based?
                  Message 8 of 21 , Feb 5, 2008
                  • 0 Attachment
                    On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:


                    Having not done scrum yet myself, I'm probably front-loading too
                    much. There will certainly be plenty of informal alignment &
                    education going on, but how does this look for the formal training
                    part?












                    Have you considered a team based approach to training rather than role based?
                  • Jon Dickinson
                    Would you run the usability test as part of the demo of the product increment? I like the idea of the demo being the usability test and the stakeholders being
                    Message 9 of 21 , Feb 5, 2008
                    • 0 Attachment
                      Would you run the usability test as part of the demo of the product increment? I like the idea of the demo being the usability test and the stakeholders being observers of the usability test.

                      > This metric would be supported by what I'm calling "commoditized
                      > user research"

                      Sounds like a blog post in the making. I would write this up outside of the list and then send a link to the list. I'm sure it would be a great way to get feedback. Not that I'm a list administrator, I've just seen people do this in the XP list and it works very well.

                      Jon.




                      On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:

                      --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                      wrote:
                      > I like your idea of building UX into the sprint exit criteria. Do
                      you have
                      > any ideas how you are going to achieve this yet?

                      My ideas so far:

                      1) Be sure exit criteria is framed from the user's perspective,
                      e.g. "Katherine (our primary person) can cancel an order." rather
                      than "implement cancel order functionality"

                      2) I think there may be some instances where it will make sense to
                      put a usability metric on the exit criteria, e.g., "Users can cancel
                      an order, and they rate their experience an average of at least 4 on
                      either the SUS or the question, "Compared to other sites, how easy
                      was this to use?"

                      This metric would be supported by what I'm calling "commoditized
                      user research"--this is something that we've started doing that sort
                      of turns most user research into a commodity--it makes it very
                      simple for the project team to get user feedback. I can say much
                      more on that if anyone's interested.


                    • Adrian Howard
                      ... That was what I was just about to suggest. Once you have an organisation that accepts that UX issues are important (and if they re financing a UX
                      Message 10 of 21 , Feb 5, 2008
                      • 0 Attachment
                        On 5 Feb 2008, at 09:04, Jon Dickinson wrote:

                        >> On 04/02/2008, timkieschnick <tim.kieschnick@...> wrote:
                        >>
                        >>>
                        >>> Having not done scrum yet myself, I'm probably front-loading too
                        >>> much. There will certainly be plenty of informal alignment &
                        >>> education going on, but how does this look for the formal training
                        >>> part?
                        >
                        > Have you considered a team based approach to training rather than role
                        > based?

                        That was what I was just about to suggest. Once you have an
                        organisation that accepts that UX issues are important (and if
                        they're financing a UX department than my guess is the answer to that
                        is yes :-) then I'd focus on educating project teams as a whole
                        rather than roles.

                        In my experience UX in agile teams is as much bottom-up as it is top-
                        down (like most other things in agile teams). I think you'll see see
                        more progress more quickly if you educate a team at a time, rather
                        than a layer that cuts across all teams.

                        Cheers,

                        Adrian
                      • Adrian Howard
                        ... [snip] I ve only managed to do it a couple of times but I find whole-team usability tests very useful. While he s not talking about an agile environment
                        Message 11 of 21 , Feb 5, 2008
                        • 0 Attachment
                          On 5 Feb 2008, at 09:53, Jon Dickinson wrote:

                          > Would you run the usability test as part of the demo of the product
                          > increment? I like the idea of the demo being the usability test and
                          > the stakeholders being observers of the usability test.
                          [snip]

                          I've only managed to do it a couple of times but I find whole-team
                          usability tests very useful. While he's not talking about an agile
                          environment Jared Spool has a nice bit on testing with large groups
                          in these two blog posts:

                          * http://www.uie.com/brainsparks/2007/09/13/usability-tests-with-30-
                          observers/
                          * http://www.uie.com/brainsparks/2007/09/18/more-on-usability-tests-
                          with-30-observers/

                          Cheers,

                          Adrian
                        • Adrian Howard
                          On 4 Feb 2008, at 22:59, timkieschnick wrote: [snip] ... [snip] While I m not a huge metric fan, if you re having problem convincing folk about the importance
                          Message 12 of 21 , Feb 5, 2008
                          • 0 Attachment
                            On 4 Feb 2008, at 22:59, timkieschnick wrote:
                            [snip]
                            > 2) I think there may be some instances where it will make sense to
                            > put a usability metric on the exit criteria, e.g., "Users can cancel
                            > an order, and they rate their experience an average of at least 4 on
                            > either the SUS or the question, "Compared to other sites, how easy
                            > was this to use?"
                            [snip]

                            While I'm not a huge metric fan, if you're having problem convincing
                            folk about the importance of UX issues having a graph of SUS numbers
                            over time in the wall along with the burndown chart, et al, can be
                            useful :-)

                            Cheers,

                            Adrian
                          • Gart, Mitchell
                            In our company we have a few UX people spread over several development teams. Each UX person has to split their time between a few development projects. Each
                            Message 13 of 21 , Feb 5, 2008
                            • 0 Attachment
                              In our company we have a few UX people spread over several development
                              teams. Each UX person has to split their time between a few development
                              projects. Each team has several full time developers and one part time
                              UX person.

                              I find that the most important thing is having a good, trusting working
                              relationship. If the developers trust the UX person, and vice versa,
                              they will be able to work together with this kind of full time - part
                              time division of labor. If they don't have a trusting relationship,
                              things will not go so well.

                              - Mitch Gart
                            • timkieschnick
                              ... and the ... Has anyone done what Jon s suggesting? Somehow combinging usability test with the demo? I like the idea of showing live user testing to the
                              Message 14 of 21 , Mar 3, 2008
                              • 0 Attachment
                                --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                                wrote:
                                >
                                > Would you run the usability test as part of the demo of the product
                                > increment? I like the idea of the demo being the usability test
                                and the
                                > stakeholders being observers of the usability test.
                                >

                                Has anyone done what Jon's suggesting? Somehow combinging usability
                                test with the demo?

                                I like the idea of showing live user testing to the stakeholders,
                                but it sounds problematic in terms of being able to show a
                                representative sample of users to the stakeholders--I wouldn't want
                                the stakeholders to see the one outlyer test, and I don't know how
                                I'd show them 4-5 users in one demo.

                                The only way I can think of to show them multiple users would be to
                                splice together video clips of representative user tests. This would
                                be fantastic in a demo, but too much work to do regularly.

                                Anyone have experience with this?
                              • Manish Pillewar
                                From a sales/business perspective: Q. Would you run the usability test as part of the demo of the product increment? Comment: It has its risks. The users
                                Message 15 of 21 , Mar 3, 2008
                                • 0 Attachment
                                  From a sales/business perspective:

                                  Q. Would you run the usability test as part of the
                                  demo of the product increment?

                                  Comment: It has its risks. The users nominated for
                                  testing need to be very close to the persona's you
                                  created during the application design. Provided you
                                  have done all the risks mitigation, its should help
                                  drive the point home asap.


                                  Q. I like the idea of the demo being the usability
                                  test
                                  and the stakeholders being observers of the usability
                                  test.

                                  Comment: The challenge here is to moderate the test
                                  show while you bring out the best features of the
                                  product. Live usability testing may bring in
                                  additional viewpoints on how the client organization
                                  works and how certain additional features would have
                                  been great to have. This could potentially become a
                                  point of negotiation on the cost estimates for your
                                  product.


                                  Q. Has anyone done what Jon's suggesting? Somehow
                                  combining usability test with the demo? I like the
                                  idea of showing live user testing to the stakeholders,
                                  but it sounds problematic in terms of being able to
                                  show a representative sample of users to the
                                  stakeholders- -I wouldn't want the stakeholders to see
                                  the one outlyer test, and I don't know how I'd show
                                  them 4-5 users in one demo. The only way I can think
                                  of to show them multiple users would be to splice
                                  together video clips of representative user tests.
                                  This would be fantastic in a demo, but too much work
                                  to do regularly.Anyone have experience with this?

                                  Comment: Showing a short demo video coupled with the
                                  statistics generated from the usability test seems to
                                  be a better idea, in my experience. It not only shows
                                  that you have taken great care while designing the
                                  product,etc. but also generates less queries and
                                  lesser debates, as the data is for all to see.

                                  I've kept aside design ethics which would otherwise
                                  dictate that in case the user testing fails, its the
                                  feedback thats important, and you want to sell usable
                                  software only, even if it means that you don't make a
                                  sale at all.





                                  Thanks and Regards
                                  Manish Govind Pillewar
                                  Sr. User Experience Designer
                                  Thoughtworks India Pvt. Ltd.Bangalore-India

                                  Tel. +91 9880566951 (M)
                                  +91 80 41113967 (Eve.)
                                  Smith & Wesson: The original point and click interface :-)



                                  ___________________________________________________________
                                  Rise to the challenge for Sport Relief with Yahoo! For Good

                                  http://uk.promotions.yahoo.com/forgood/
                                • Jeff Lopez-Stuit
                                  I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo
                                  Message 16 of 21 , Mar 3, 2008
                                  • 0 Attachment
                                    I think it would be much more meaningful to the product owners if you could demonstrate the *value* of the testing you did during the sprint, rather than demo the testing itself.   The objective of the demo should be to show what value was produced, not how the work was done.  You can provide a quick summary the tests, but the focus should be on the specific measurable improvements in usability that were identified and addressed during the sprint.

                                    Take care...

                                    Jeff Lopez-Stuit, CSM, CSPO
                                    SolutionsIQ

                                    On Mon, Mar 3, 2008 at 3:14 PM, timkieschnick <tim.kieschnick@...> wrote:

                                    --- In agile-usability@yahoogroups.com, "Jon Dickinson" <jon@...>
                                    wrote:
                                    >
                                    > Would you run the usability test as part of the demo of the product
                                    > increment? I like the idea of the demo being the usability test
                                    and the
                                    > stakeholders being observers of the usability test.
                                    >


                                    .


                                  • Jon Dickinson
                                    On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit ... I am making the assumption that the product owners already understand the value of usability testing,
                                    Message 17 of 21 , Mar 8, 2008
                                    • 0 Attachment
                                      On Tue, 04 Mar 2008 07:03:09 -0000, Jeff Lopez-Stuit
                                      <jeff.lopezstuit@...> wrote:

                                      > I think it would be much more meaningful to the product owners if you
                                      > could
                                      > demonstrate the *value* of the testing you did during the sprint, rather
                                      > than demo the testing itself.

                                      I am making the assumption that the product owners already understand the
                                      value of usability testing, so it is not necessary to demonstrate the
                                      value of the technique.

                                      > The objective of the demo should be to show
                                      > what value was produced, not how the work was done.

                                      Agreed. The product managers attend a usability test not to see how a
                                      usability test is performed but to shorten the feedback loop from the
                                      results of the test.

                                      > You can provide a quick
                                      > summary the tests, but the focus should be on the specific measurable
                                      > improvements in usability that were identified and addressed during the
                                      > sprint.


                                      Over the course of a few end of sprint usability tests the product
                                      managers should get a good first hand impression of whether usability
                                      improvements are being made during the sprints. This doesn't stop one
                                      keeping records of the tests and producing measurements to show usability
                                      improvements over time, if this provides any additional value.

                                      --
                                      Jon Dickinson
                                      Accolade Consulting
                                      http://www.accolade-consulting.co.uk
                                    • Jon Dickinson
                                      On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar ... Is this not a useful exercise to validate your personas are correct? ... Assuming your product has more
                                      Message 18 of 21 , Mar 8, 2008
                                      • 0 Attachment
                                        On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
                                        <manish1022@...> wrote:

                                        >
                                        >
                                        >
                                        > From a sales/business perspective:
                                        >
                                        > Q. Would you run the usability test as part of the
                                        > demo of the product increment?
                                        >
                                        > Comment: It has its risks. The users nominated for
                                        > testing need to be very close to the persona's you
                                        > created during the application design.

                                        Is this not a useful exercise to validate your personas are correct?

                                        >
                                        > Q. I like the idea of the demo being the usability
                                        > test
                                        > and the stakeholders being observers of the usability
                                        > test.
                                        >
                                        > Comment: The challenge here is to moderate the test
                                        > show while you bring out the best features of the
                                        > product. Live usability testing may bring in
                                        > additional viewpoints on how the client organization
                                        > works and how certain additional features would have
                                        > been great to have. This could potentially become a
                                        > point of negotiation on the cost estimates for your
                                        > product.

                                        Assuming your product has more than one sprint, and therefore more than
                                        one usability test, before releasing why would this be a bad thing?

                                        > Comment: Showing a short demo video coupled with the
                                        > statistics generated from the usability test seems to
                                        > be a better idea, in my experience. It not only shows
                                        > that you have taken great care while designing the
                                        > product,etc. but also generates less queries and
                                        > lesser debates, as the data is for all to see.

                                        Why would you want to reduce debate? The idea of running a usibility test
                                        at the end of the sprint is generate discussion to help shape the work for
                                        the next sprint.

                                        --
                                        Jon Dickinson
                                        Accolade Consulting
                                        http://www.accolade-consulting.co.uk
                                      • Jon Dickinson
                                        On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick ... I can see that it would be a risk if the results of one bad test/demonstration to
                                        Message 19 of 21 , Mar 8, 2008
                                        • 0 Attachment
                                          On Mon, 03 Mar 2008 23:14:03 -0000, timkieschnick <tim.kieschnick@...>
                                          wrote:

                                          > I like the idea of showing live user testing to the stakeholders,
                                          > but it sounds problematic in terms of being able to show a
                                          > representative sample of users to the stakeholders--I wouldn't want
                                          > the stakeholders to see the one outlyer test, and I don't know how
                                          > I'd show them 4-5 users in one demo.

                                          I can see that it would be a risk if the results of one bad
                                          test/demonstration to the stakeholders sends the team off in the wrong
                                          direction for an entire sprint.

                                          From reading your excellent experience report:
                                          http://timiti.blogspot.com/2008/02/user-research-as-commodity.html, the
                                          following passage strikes me as providing two possible solutions:

                                          "Every other Thursday morning, four or five brokers (our target audience)
                                          would show up at our offices in Pasadena. Our user research specialist
                                          would work with each participant for about an hour. While this testing
                                          took place in Pasadena, we piped audio and video of the testing up north
                                          to Pleasanton so the team could watch in real-time and IM questions and
                                          comments to the moderator in Pasadena."

                                          1) Persuade the stakeholders to spend the entire morning at the end of the
                                          sprint observing the usability tests. This is my ideal.
                                          2) Have the stakeholders sit in as observers on the test with the last
                                          participant. If the results of this test turn out to be radically
                                          different to the earlier tests, you have the results of the previous tests
                                          to demonstrate that this was the exception.

                                          If the stakeholders are limited to observing a single test, it is probably
                                          worth giving them a half hour demo of the work completed before the
                                          usability test so they have an initial idea of what work has been
                                          completed. You don't want one bad test to negate all the hard work the
                                          team has performed during the sprint.

                                          Cheers,

                                          --
                                          Jon Dickinson
                                          Accolade Consulting
                                          http://www.accolade-consulting.co.uk
                                        • Manish Pillewar
                                          Hi Jon, Your questions are of course valid from a designers perspective.However,my comment have been from the sales/business perspective. refer the first
                                          Message 20 of 21 , Mar 9, 2008
                                          • 0 Attachment
                                            Hi Jon,
                                            Your questions are of course valid from a designers
                                            perspective.However,my comment have been from the
                                            sales/business perspective. refer the first statement
                                            in my email

                                            "> From a sales/business perspective:"


                                            Hope you see that perspective.
                                            -Manish


                                            On Tue, 04 Mar 2008 04:47:27 -0000, Manish Pillewar
                                            <manish1022@...> wrote:

                                            >
                                            >
                                            >
                                            > From a sales/business perspective:
                                            >
                                            > Q. Would you run the usability test as part of the
                                            > demo of the product increment?
                                            >
                                            > Comment: It has its risks. The users nominated for
                                            > testing need to be very close to the persona's you
                                            > created during the application design.

                                            Is this not a useful exercise to validate your
                                            personas are correct?

                                            >
                                            > Q. I like the idea of the demo being the usability
                                            > test
                                            > and the stakeholders being observers of the
                                            usability
                                            > test.
                                            >
                                            > Comment: The challenge here is to moderate the test
                                            > show while you bring out the best features of the
                                            > product. Live usability testing may bring in
                                            > additional viewpoints on how the client organization
                                            > works and how certain additional features would have
                                            > been great to have. This could potentially become a
                                            > point of negotiation on the cost estimates for your
                                            > product.

                                            Assuming your product has more than one sprint, and
                                            therefore more than

                                            one usability test, before releasing why would this be
                                            a bad thing?

                                            > Comment: Showing a short demo video coupled with the
                                            > statistics generated from the usability test seems
                                            to
                                            > be a better idea, in my experience. It not only
                                            shows
                                            > that you have taken great care while designing the
                                            > product,etc. but also generates less queries and
                                            > lesser debates, as the data is for all to see.

                                            Why would you want to reduce debate? The idea of
                                            running a usibility
                                            test
                                            at the end of the sprint is generate discussion to
                                            help shape the work
                                            for
                                            the next sprint.

                                            --
                                            Jon Dickinson
                                            Accolade Consulting
                                            http://www.accolade-consulting.co.uk

                                            Thanks and Regards
                                            Manish Govind Pillewar
                                            Sr. User Experience Designer
                                            Thoughtworks India Pvt. Ltd.Bangalore-India

                                            Tel. +91 9880566951 (M)
                                            +91 80 41113967 (Eve.)
                                            Smith & Wesson: The original point and click interface :-)



                                            __________________________________________________________
                                            Sent from Yahoo! Mail.
                                            The World's Favourite Email http://uk.docs.yahoo.com/nowyoucan.html
                                          • Jon Dickinson
                                            On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar ... Hi Manesh, Are you talking about a situation where the business/sales goals are at odds with the goals
                                            Message 21 of 21 , Mar 10, 2008
                                            • 0 Attachment
                                              On Mon, 10 Mar 2008 06:10:46 -0000, Manish Pillewar
                                              <manish1022@...> wrote:

                                              > Hi Jon,
                                              > Your questions are of course valid from a designers
                                              > perspective.However,my comment have been from the
                                              > sales/business perspective. refer the first statement
                                              > in my email
                                              >
                                              > "> From a sales/business perspective:"
                                              >
                                              >
                                              > Hope you see that perspective.
                                              > -Manish
                                              >

                                              Hi Manesh,

                                              Are you talking about a situation where the business/sales goals are at
                                              odds with the goals of the rest of the team?

                                              If that is the case then there is a bigger problem than trying to reduce
                                              the feedback loop for usability testing. Although having stakeholders sit
                                              in on the usability tests, but not as part of the end of sprint demo might
                                              still help to address this.

                                              If there is a misalignment between the goals of the stakeholders and the
                                              implementation team then I would agree that this technique could cause
                                              issues. There would need to be a certain amount of trust and an open
                                              working relationship between the team and the stakeholders for this to
                                              work.

                                              Thanks,

                                              --
                                              Jon Dickinson
                                              Accolade Consulting Ltd.
                                              http://www.accolade-consulting.co.uk
                                              http://www.planitagile.com
                                            Your message has been successfully submitted and would be delivered to recipients shortly.