Loading ...
Sorry, an error occurred while loading the content.

How do you avoid scope creep when integrating user research into the scrums?

Expand Messages
  • Jared Spool
    Hi there, I have a client who recently asked me about this problem. I was wondering how you d handle it? They are new to UX and to Agile methods, so they are
    Message 1 of 13 , Aug 11, 2011
    View Source
    • 0 Attachment
      Hi there,

      I have a client who recently asked me about this problem. I was wondering how you'd handle it?

      They are new to UX and to Agile methods, so they are struggling in interesting ways.

      As part of a recent project, they added some usability testing to their sprints. They'd never done a usability test before and, as a result, discovered many usability issues with the design.

      However, many of the problems they found were outside the original scope of the project. The UX team members wanted desperately to add many of the problems to the project's backlog, but instantly found resistance from the product owners who didn't want to delay the delivery. The result was a sense from the UX folks that the product owners didn't care about a good user experience.

      My question is how might you have avoided this problem? I had initially suggested that the team needed more upfront definition of what is and isn't within the project's scope and a way to record outside-of-scope usability issues for future projects.

      What do you think?

      Jared

      Jared M. Spool
      User Interface Engineering
      510 Turnpike St., Suite 102, North Andover, MA 01845
      e: jspool@... p: +1 978 327 5561
      http://uie.com Blog: http://uie.com/brainsparks Twitter: @jmspool
    • Jeff Wall
      I think the answer depends on whose perspective you are viewing this from. From the Product Owners perspective, it is the teams job to do exactly what they
      Message 2 of 13 , Aug 11, 2011
      View Source
      • 0 Attachment
        I think the answer depends on whose perspective you are viewing this from.

        From the Product Owners perspective, it is the teams job to do exactly what they did.   They ran the tests got the results and presented them to the Product Owners.  It is the Product Owners job to prioritize the Stories.  If basic functionality trumps a better design that is their call.

        From the development teams perspective is sounds like the Product Owners could use some Agile Coaching.  Change in an Agile project is a good thing and should not be considered scope creep.

        As a User experience Analyst and a Business Analyst I've been on a project where we had to deliver a large amount of basic functionality in a given time frame (the time frame was our mistake).  So, we didn't have room in the iterations for stories to modify the UI the way it needed and incorporate feedback.  It had to wait until phase 2.....

        Jeffrey M. Wall
        Sr. Business Systems Analyst
        jmwall67@...

        On Thu, Aug 11, 2011 at 9:16 PM, Jared Spool <jspool@...> wrote:
         

        Hi there,

        I have a client who recently asked me about this problem. I was wondering how you'd handle it?

        They are new to UX and to Agile methods, so they are struggling in interesting ways.

        As part of a recent project, they added some usability testing to their sprints. They'd never done a usability test before and, as a result, discovered many usability issues with the design.

        However, many of the problems they found were outside the original scope of the project. The UX team members wanted desperately to add many of the problems to the project's backlog, but instantly found resistance from the product owners who didn't want to delay the delivery. The result was a sense from the UX folks that the product owners didn't care about a good user experience.

        My question is how might you have avoided this problem? I had initially suggested that the team needed more upfront definition of what is and isn't within the project's scope and a way to record outside-of-scope usability issues for future projects.

        What do you think?

        Jared

        Jared M. Spool
        User Interface Engineering
        510 Turnpike St., Suite 102, North Andover, MA 01845
        e: jspool@... p: +1 978 327 5561
        http://uie.com Blog: http://uie.com/brainsparks Twitter: @jmspool




        --

      • Prasanna Prabhu
        Jared,   They need to understand 1 thing for sure, if there are no users willing to use their product that they are building, then they are wasting their time
        Message 3 of 13 , Aug 11, 2011
        View Source
        • 0 Attachment
          Jared,
           
          They need to understand 1 thing for sure, if there are no users willing to use their product that they are building, then they are wasting their time
          Not that I'm saying Usability is everything, but for sure not paying attention to it will cost a ton as the tail-end of the project.
          Most easy way to solve the problem is involve en-users in the product building mix as they will tell how easy it is for them to use who in-tirn will be your spokesperson for rest of the customer...
           
          Prasanna Prabhu
          From: Jared Spool <jspool@...>
          To: agile-usability@yahoogroups.com
          Sent: Thursday, August 11, 2011 6:16 PM
          Subject: [agile-usability] How do you avoid scope creep when integrating user research into the scrums?

          Hi there,

          I have a client who recently asked me about this problem. I was wondering how you'd handle it?

          They are new to UX and to Agile methods, so they are struggling in interesting ways.

          As part of a recent project, they added some usability testing to their sprints. They'd never done a usability test before and, as a result, discovered many usability issues with the design.

          However, many of the problems they found were outside the original scope of the project. The UX team members wanted desperately to add many of the problems to the project's backlog, but instantly found resistance from the product owners who didn't want to delay the delivery. The result was a sense from the UX folks that the product owners didn't care about a good user experience.

          My question is how might you have avoided this problem? I had initially suggested that the team needed more upfront definition of what is and isn't within the project's scope and a way to record outside-of-scope usability issues for future projects.

          What do you think?

          Jared

          Jared M. Spool
          User Interface Engineering
          510 Turnpike St., Suite 102, North Andover, MA 01845
          e: jspool@... p: +1 978 327 5561
          http://uie.com  Blog: http://uie.com/brainsparks  Twitter: @jmspool




          ------------------------------------

          Yahoo! Groups Links

          <*> To visit your group on the web, go to:
              http://groups.yahoo.com/group/agile-usability/

          <*> Your email settings:
              Individual Email | Traditional

          <*> To change settings online go to:
              http://groups.yahoo.com/group/agile-usability/join
              (Yahoo! ID required)

          <*> To change settings via email:
              agile-usability-digest@yahoogroups.com
              agile-usability-fullfeatured@yahoogroups.com

          <*> To unsubscribe from this group, send an email to:
              agile-usability-unsubscribe@yahoogroups.com

          <*> Your use of Yahoo! Groups is subject to:
              http://docs.yahoo.com/info/terms/



           saying
        • Marcy Jacobs
          How significant are the usability issues? Is a few extra clicks and not the most elegant design but the user can complete tasks or are these issues preventing
          Message 4 of 13 , Aug 11, 2011
          View Source
          • 0 Attachment
            How significant are the usability issues? Is a few extra clicks and not the most elegant design but the user can complete tasks or are these issues preventing the user from accomplishing critical tasks?

            There are always competing priorities and too many things to fit into a sprint but if the issues are serious enough, their business value should get them included.

            Marcy

            Sent from my iPad
          • Austin Govella
            On Aug 11, 2011, at 8:16 PM, Jared Spool wrote: I have a client who recently asked me about this problem. I was wondering how you d handle it?
            Message 5 of 13 , Aug 11, 2011
            View Source
            • 0 Attachment
              On Aug 11, 2011, at 8:16 PM, Jared Spool <jspool@...> wrote:
               

              I have a client who recently asked me about this problem. I was wondering how you'd handle it?

              They are new to UX and to Agile methods, so they are struggling in interesting ways.

              As part of a recent project, they added some usability testing to their sprints. They'd never done a usability test before and, as a result, discovered many usability issues with the design.

              However, many of the problems they found were outside the original scope of the project. The UX team members wanted desperately to add many of the problems to the project's backlog, but instantly found resistance from the product owners who didn't want to delay the delivery. The result was a sense from the UX folks that the product owners didn't care about a good user experience.

              The most important thing to remember is that agile changes product development into a continuously moving stream, and you have to be willing to jump in and float along, changing what you can when you can.

              Specifically, though there are lots of small ways to mitigate this problem:

              1. The product backlog should be prioritized. Any new work is added to the product backlog and prioritized accordingly. If the usability fixes get prioritized at the bottom of the pile, then so be it.

              2. UX Bucket. Have the teams begin setting aside a set number of developer hours/points specifically for the UX team to use. These can be used for usability fixes that don't make it high enough on the backlog. Or to do research. Or to do fixes you notice.

              3. The Shelf. Stick the fixes "on the shelf". Next time you're developing in that area of the product, pull them down and roll the changes into adjacent work. (A lot of changes are absolutely negligible if a developer is already in that neighborhood of the code.)

              4. Schedule iteration/fix time into sprints. A test requires time for testing and time for fixes. Of course, no one will know what the fixes will be until the tests are done, so they may grouse about scheduling a black box of time (identical to the aforementioned UX bucket), but they need to suck it up. :-)

              5. Defining UX quality of 'done'. Disagreements about what needs to be completed before you can deliver is really a disagreement about what done means. UX quality should be part of the definition of done. When you schedule a story, everyone on the team should agree whether you're shooting for functional, usable, parity with comparable experiences, or awesome,

              6. Writer better stories. Add comparisons. (Registration should be like Twitter's.)

              7. Publicize how bad the usability errors are to everyone at the company. Commonly perceived problems are more quickly addressed than problems only you see. (Chicken Little never gets the sky fixed unless he's the CEO or product manager.)

              8. Design mockups or prototypes of the fix and publicize. It's easier to squeeze in additional scope when it's super clear what the change is. And/or stick these fixes on the walls. Sooner or later, people will wonder why these completed fixes haven't been implemented. For goodness sakes, they're already designed!

              9. Measure key baselines. If the fixes would effect key baselines, then you have a stronger case for getting them prioritized towards the top of the backlog.

              10. Forward (or print and post, drop on desks) any case studies you see where usability fixes and/or ux improvements improved metrics. No one cares about UX. Everyone cares about success.



              --
              Austin
            • Jared Spool
              ... Good question about significance. The usability issues were significant, but not for things the product owner wanted to focus on. Because this was the
              Message 6 of 13 , Aug 11, 2011
              View Source
              • 0 Attachment

                On Aug 11, 2011, at 10:03 PM, Marcy Jacobs wrote:

                 

                How significant are the usability issues? Is a few extra clicks and not the most elegant design but the user can complete tasks or are these issues preventing the user from accomplishing critical tasks?

                There are always competing priorities and too many things to fit into a sprint but if the issues are serious enough, their business value should get them included.

                Good question about significance.

                The usability issues were significant, but not for things the product owner wanted to focus on. Because this was the first time they'd ever done serious usability testing, they found a ton of stuff wrong all over the place.

                The fear was if they added all the work to the backlog and prioritized it as critical, the original scoped work would never get completed enough to ship.

                The team is still wrestling with both how to prioritize things (not just usability related) that crop up and how to factor UX into the process.

                Jared

              • Jared Spool
                ... You re absolutely correct. The interesting fact is that the product is in use by thousands of customers today. The users have a love/hate relationship with
                Message 7 of 13 , Aug 11, 2011
                View Source
                • 0 Attachment

                  On Aug 11, 2011, at 9:52 PM, Prasanna Prabhu wrote:

                  They need to understand 1 thing for sure, if there are no users willing to use their product that they are building, then they are wasting their time
                  Not that I'm saying Usability is everything, but for sure not paying attention to it will cost a ton as the tail-end of the project.

                  You're absolutely correct.

                  The interesting fact is that the product is in use by thousands of customers today. The users have a love/hate relationship with it. The product has been successful in its market despite itself, mostly because of sophisticated (albeit hard to use) capabilities and no real competitors (until recently).

                  The market factors are changing, thus the new focus on agile and on UX. This is a new world for these guys.

                  Jared


                • Yaniv Nord
                  It s virtually impossible to do usability testing in the context of a sprint. By the time your story is pointed, commitment line set, and sprint has started,
                  Message 8 of 13 , Aug 11, 2011
                  View Source
                  • 0 Attachment
                    It's virtually impossible to do usability testing in the context of a sprint.

                    By the time your story is pointed, commitment line set, and sprint has
                    started, you should have your ux work 95% stable.

                    That means your work (research, design, testing, iteration, etc) needs
                    to happen up front, running at least one sprint ahead, preferably two
                    or three. I have yet to talk to anyone doing ux in an serious agile
                    environment that's able to iterate design in any meaningful way during
                    a sprint.

                    It's just massively disruptive, and I would say even disrespectful, to
                    come back mid-sprint to your developers and QA team on a story they've
                    already signed off on with, "oh hey by the way our design doesn't work
                    so we need to change it all."



                    On Thu, Aug 11, 2011 at 9:16 PM, Jared Spool <jspool@...> wrote:
                    > Hi there,
                    >
                    > I have a client who recently asked me about this problem. I was wondering how you'd handle it?
                    >
                    > They are new to UX and to Agile methods, so they are struggling in interesting ways.
                    >
                    > As part of a recent project, they added some usability testing to their sprints. They'd never done a usability test before and, as a result, discovered many usability issues with the design.
                    >
                    > However, many of the problems they found were outside the original scope of the project. The UX team members wanted desperately to add many of the problems to the project's backlog, but instantly found resistance from the product owners who didn't want to delay the delivery. The result was a sense from the UX folks that the product owners didn't care about a good user experience.
                    >
                    > My question is how might you have avoided this problem? I had initially suggested that the team needed more upfront definition of what is and isn't within the project's scope and a way to record outside-of-scope usability issues for future projects.
                    >
                    > What do you think?
                    >
                    > Jared
                    >
                    > Jared M. Spool
                    > User Interface Engineering
                    > 510 Turnpike St., Suite 102, North Andover, MA 01845
                    > e: jspool@... p: +1 978 327 5561
                    > http://uie.com  Blog: http://uie.com/brainsparks  Twitter: @jmspool
                    >
                    >
                    >
                    >
                    > ------------------------------------
                    >
                    > Yahoo! Groups Links
                    >
                    >
                    >
                    >
                  • Adrian Howard
                    Hi Jared, On 12 Aug 2011, at 02:16, Jared Spool wrote: [snip] ... Not a problem unique to agile. I m amazed at the number of folk who budget in time for
                    Message 9 of 13 , Aug 12, 2011
                    View Source
                    • 0 Attachment
                      Hi Jared,

                      On 12 Aug 2011, at 02:16, Jared Spool wrote:
                      [snip]
                      > However, many of the problems they found were outside the original scope of the project. The UX team members wanted desperately to add many of the problems to the project's backlog, but instantly found resistance from the product owners who didn't want to delay the delivery. The result was a sense from the UX folks that the product owners didn't care about a good user experience.

                      Not a problem unique to agile. I'm amazed at the number of folk who budget in time for usability testing, but expect the findings can magically be integrated with no additional work :-)

                      > My question is how might you have avoided this problem? I had initially suggested that the team needed more upfront definition of what is and isn't within the project's scope and a way to record outside-of-scope usability issues for future projects.
                      [snip]

                      I think your initial suggestion is a good one. Before I start doing user testing with a team we try and figure out what is/is not possible implementation wise. This is good for two reasons:

                      1) It helps set expectations on both sides of what is going to be done with the results. So I'm not left being annoyed that my sage advice is being ignored, and the team isn't given a set of high level issues that cannot be fixed in the scope of the current release plan.

                      2) It allows better targeting of the UX resources. If we know that we're not realistically have time to completely re-implement feature Foo, then we can spend more time looking at feature Bar instead - or more time teaching/facilitating good UX work elsewhere.

                      In getting teams to value UX I find it *much* more effective to provide advice that can be acted on immediately so the value of that advice can be quickly seen. Once you start showing value it's much easier to get agreement on addressing larger issues.

                      Having that conversation about how the usability testing can effect the end product can lead to some interesting places.

                      For example it may highlight situations where the product development team needs to become more agile.

                      How fixed is the release plan? Is it fixed for a good reason? Is new feature Foo really more important than fixing feature Bar? Is agreeing a release plan N months in advance helping or hindering building a better product?

                      The conversation about how usability testing results fit in can rapidly become a much larger discussion about the process as a whole, the definition of done, and how features are prioritised.

                      This larger conversation is useful and good - but can sometimes get in the way of addressing the short term issues that can feasibly be addressed in the context of the next couple of iterations. Especially, like your client, where you're facing a huge usability debt.

                      Addressing the small issues first. Show that they help. Use that evidence as the fuel to drive addressing larger scale issues.

                      Cheers,

                      Adrian
                      --
                      http://quietstars.com adrianh@... twitter.com/adrianh
                      t. +44 (0)7752 419080 skype adrianjohnhoward del.icio.us/adrianh
                    • Adrian Howard
                      On 12 Aug 2011, at 03:05, Austin Govella wrote: [snip] ... [snip] This is really good advice - and is often one that is easy for developers to understand since
                      Message 10 of 13 , Aug 12, 2011
                      View Source
                      • 0 Attachment
                        On 12 Aug 2011, at 03:05, Austin Govella wrote:
                        [snip]
                        > 3. The Shelf. Stick the fixes "on the shelf". Next time you're developing in that area of the product, pull them down and roll the changes into adjacent work. (A lot of changes are absolutely negligible if a developer is already in that neighborhood of the code.)
                        [snip]

                        This is really good advice - and is often one that is easy for developers to understand since it's the same approach many teams take when addressing technical debt issues.

                        Cheers,

                        Adrian
                        --
                        http://quietstars.com adrianh@... twitter.com/adrianh
                        t. +44 (0)7752 419080 skype adrianjohnhoward del.icio.us/adrianh
                      • Adrian Howard
                        ... I think that depends on the kind of usability testing that you re doing, and the issues that your product development process is leaving for you to
                        Message 11 of 13 , Aug 12, 2011
                        View Source
                        • 0 Attachment
                          On 12 Aug 2011, at 03:53, Yaniv Nord wrote:

                          > It's virtually impossible to do usability testing in the context of a sprint.
                          >
                          > By the time your story is pointed, commitment line set, and sprint has
                          > started, you should have your ux work 95% stable.
                          >
                          > That means your work (research, design, testing, iteration, etc) needs
                          > to happen up front, running at least one sprint ahead, preferably two
                          > or three. I have yet to talk to anyone doing ux in an serious agile
                          > environment that's able to iterate design in any meaningful way during
                          > a sprint.

                          I think that depends on the kind of usability testing that you're doing, and the issues that your product development process is leaving for you to discover mid-sprint.

                          There are people successfully doing in-sprint usability testing - even multiple sets of usability tests during a single sprint (see http://is.gd/h2YpDq for example).

                          The key difference I see with teams that have that approach are:

                          * they've got a product development process that's leaving them with few, if any, large usability issues to discover at that late stage

                          * they have a very clear definition of "done" that the whole team values so they can quickly decide whether an issue needs to be addressed immediately or postponed to a later iteration

                          Cheers,

                          Adrian
                          --
                          http://quietstars.com adrianh@... twitter.com/adrianh
                          t. +44 (0)7752 419080 skype adrianjohnhoward del.icio.us/adrianh
                        • Jeff Wall
                          Adrian, That is a good point about the Definition of Done At the beginning of our project we wrote down our teams Definition of Done and posted it on the
                          Message 12 of 13 , Aug 12, 2011
                          View Source
                          • 0 Attachment
                            Adrian,

                            That is a good point about the "Definition of Done"

                            At the beginning of our project we wrote down our teams Definition of Done and posted it on the wall where we do our stand-ups so we could see it every day.

                            It has about 10 items on it.  Some for developers (i.e. full unit test coverage), some for QA, and UI/UX.

                            It is amazing how difficult it is to complete an iteration successfully, with a high degree of quality and usability when you have a documented set of items that define "Done."

                            I highly recommend this practice.

                            Jeff
                             

                            On Fri, Aug 12, 2011 at 6:05 AM, Adrian Howard <adrianh@...> wrote:
                             


                            On 12 Aug 2011, at 03:53, Yaniv Nord wrote:

                            > It's virtually impossible to do usability testing in the context of a sprint.
                            >
                            > By the time your story is pointed, commitment line set, and sprint has
                            > started, you should have your ux work 95% stable.
                            >
                            > That means your work (research, design, testing, iteration, etc) needs
                            > to happen up front, running at least one sprint ahead, preferably two
                            > or three. I have yet to talk to anyone doing ux in an serious agile
                            > environment that's able to iterate design in any meaningful way during
                            > a sprint.

                            I think that depends on the kind of usability testing that you're doing, and the issues that your product development process is leaving for you to discover mid-sprint.

                            There are people successfully doing in-sprint usability testing - even multiple sets of usability tests during a single sprint (see http://is.gd/h2YpDq for example).

                            The key difference I see with teams that have that approach are:

                            * they've got a product development process that's leaving them with few, if any, large usability issues to discover at that late stage

                            * they have a very clear definition of "done" that the whole team values so they can quickly decide whether an issue needs to be addressed immediately or postponed to a later iteration




                            --
                            Jeffrey M. Wall
                            Sr. Business Systems Analyst
                            919.355.8312
                            jmwall67@...
                          • Michael James
                            Yeah, I m also not buying that it s virtually impossible to do usability testing during the Sprint, since I ve seen people doing it. It requires different
                            Message 13 of 13 , Aug 12, 2011
                            View Source
                            • 0 Attachment
                              Yeah, I'm also not buying that it's virtually impossible to do usability testing during the Sprint, since I've seen people doing it.  It requires different habits that will initially feel inefficient, and drives us toward different technologies. But we already knew that was part of the Agile pathway. 

                              Users are becoming less tolerant of usability problems, so we need to discover these early, like all other defects. 

                              On Aug 12, 2011, at 3:05 AM, Adrian Howard <adrianh@...> wrote:

                               


                              On 12 Aug 2011, at 03:53, Yaniv Nord wrote:

                              > It's virtually impossible to do usability testing in the context of a sprint.
                              >
                              > By the time your story is pointed, commitment line set, and sprint has
                              > started, you should have your ux work 95% stable.
                              >
                              > That means your work (research, design, testing, iteration, etc) needs
                              > to happen up front, running at least one sprint ahead, preferably two
                              > or three. I have yet to talk to anyone doing ux in an serious agile
                              > environment that's able to iterate design in any meaningful way during
                              > a sprint.

                              I think that depends on the kind of usability testing that you're doing, and the issues that your product development process is leaving for you to discover mid-sprint.

                              There are people successfully doing in-sprint usability testing - even multiple sets of usability tests during a single sprint (see http://is.gd/h2YpDq for example).

                              The key difference I see with teams that have that approach are:

                              * they've got a product development process that's leaving them with few, if any, large usability issues to discover at that late stage

                              * they have a very clear definition of "done" that the whole team values so they can quickly decide whether an issue needs to be addressed immediately or postponed to a later iteration

                              Cheers,

                              Adrian
                              --
                              http://quietstars.com adrianh@... twitter.com/adrianh
                              t. +44 (0)7752 419080 skype adrianjohnhoward del.icio.us/adrianh

                            Your message has been successfully submitted and would be delivered to recipients shortly.