Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Re: UXI Matrix

Expand Messages
  • Austin Govella
    On Mon, Feb 6, 2012 at 2:10 PM, kerrykimbrough ... Kerry, you re right about the difficulty of using the UXI matrix inside existing agile software tools, but a
    Message 1 of 15 , Feb 7, 2012
    • 0 Attachment
      On Mon, Feb 6, 2012 at 2:10 PM, kerrykimbrough
      <kerry@...> wrote:
      > Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've ever seen
      > would use it. No one wants to fuss with this much analysis and data entry. And, if the
      > team is using Rally or similar tracking tools, there is zero interest in data that's not in
      > the tool.

      Kerry, you're right about the difficulty of using the UXI matrix
      inside existing agile software tools, but a lot of the teams I work
      with have abandoned the tools because they suck. That means I spend a
      lot more time in Excel than is healthy.

      However, I've morphed the product backlog in similar ways to Jon's
      matrix and it works fine. The trick to the UXI matrix isn't the
      additional analysis it seems to require, but that it puts UX analysis
      you *should* be doing anyway in the backlog adjacent to the stories.

      For example, the persona is marked for each story. Other teams put the
      persona name in the story. And he's added the UX definition of "done"
      (tested, how research-based) as a column next to where the devs track
      they're done.

      And you're right about devs not tracking the info they don't care
      about. Putting behind us how immature and rude that behavior is, I've
      compensated by creating the initial backlog. That gives me control
      over what columns are included. And when I can, I manage the backlog
      myself. I haven't met anyone who minds me owning or updating backlog
      info.



      > My 2nd reaction is that this matrix misses the point. What we need is good flow. We
      > need work to arrive at each point in the flow completely ready for the next step. Devs
      > don't want to fuss with your measures of UI design readiness. They want it to be ready.

      That's their problem: short-sighted and doomed to failure. Either
      everyone on the team can engage with everyone else and respect
      everyone's individual needs and concerns, or you've identified an
      asshole on the team.



      > We already know the waterfall flow, with its big hopeless handoffs, doesn't work.
      > Instead, we seek a more continous and incremental flow. But I don't see how the UXI
      > matrix contributes.

      Agile's biggest failing is that it usually trades waterfall's large
      hopeless handoffs for smaller hopeless handoffs, more quickly, that
      are shippable.

      Tracking extra columns in your backlog won't interrupt your flow. And
      if they help move your team towards a better definition of done, then
      they're moving the team towards better software.



      So, here's the constructive part: what information would your perfect
      product backlog track?



      --
      Austin Govella
      User Experience



      Catch me at the IA Summit in New Orleans, Mar 23-25, presenting
      "The miracle farmer's almanac":
      * http://2012.iasummit.org/schedule/miracle_farmers_almanac.html

      Watch my presentation from Schiplcon 2011:
      * http://schipul.com/videos/user-experience-and-power-great-design-austin-gove/

      Listen to my presentation from SXSWi 2011:
      * http://schedule.sxsw.com/events/event_IAP7545

      Work: http://www.agux.co
      Blog: http://www.thinkingandmaking.com

      ag@...
    • Tom Hume
      It strikes me that what you re tracking here is the state of individual backlog items. In your case, it s whether they re ready to move to development by
      Message 2 of 15 , Feb 7, 2012
      • 0 Attachment
        It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.

        I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

        This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

        Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

        On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

        So, here's the constructive part: what information would your perfect
        product backlog track?




        --
        Future Platforms: hungry and foolish since 2000
        work: Tom.Hume@... play: tomhume.org both: +447971781422

      • Jon Innes
        Kim, I guess your experience is very different than mine. While I agree with your point about teams using one tool, the existing tools don t really capture
        Message 3 of 15 , Feb 7, 2012
        • 0 Attachment
          Kim,

          I guess your experience is very different than mine.

          While I agree with your point about teams using one tool, the existing tools don't really capture this type of information and that's a problem. I've helped teams customize tools to include some of this type of stuff, but that can be a pain. It's often easier to just export to Excel or copy and paste. I've talked to a couple of tool vendors about making this easier, but none do so far.

          In keeping with agile best practices I try to avoid being constrained by tools that don't do what is useful. I choose tools based on what I want to do, not the other way around. You say there is "zero interest in data not in the tool", I guess where you work nobody cares about UX metrics. I prefer to work at places where that's not the case. In fact, I've found printing a big copy of the UXI Matrix and hanging it up generates a lot of interest, and it drives teams to be aware of UX. Just like posting a burndown chart or using a physical Scrum board helps teams new to agile "get it."

          I've also found it to be the case that a lot of teams still use Excel, just as Austin mentioned in his reply. In fact, I'd say it's the most common tool I've encountered, largely because a lot of product owners find it easier to create the initial list of stories in Excel. I first came up with the idea of listing the personas/users in columns because it drove me nuts to repeatedly enter the name of the user for many stories. Using this format saves me time when creating or updating backlogs in sprint planning meetings, which is really where it counts. 

          I agree that flow is important. When I first rolled out Scrum in a big organization, the UI designers couldn't keep up at first. We didn't do sprint 0s, and the developers were all complaining. I used an early version of the UXI Matrix to prioritize work, make it clear where the flow was broken, and to make sure the team was all on the same page about the big picture. In this case it was "sorry, you won't get a detailed spec for all stories, maybe we can pair up or just do something lightweight." We also had a lot of designers complaining that what we built sucked, because we did lightweight (or no) designs, and the developers were only focused on what their tool tracked, velocity of coding tasks.

          The idea of the UXI matrix is to help folks map the design and research work into smaller more agile portions, and then provide the same level of visibility for UX work as development work. It also makes it easy to see false velocity. Hmm, lots of iterations, but user satisfaction is the same, or usability is actually getting worse...

          Part of getting teams to understand the value of UX is making this stuff more visible. Google Analytics did wonders in terms of educating many folks about the power of data, yet it's a separate tool too. The lean startup crowd gets this stuff, they know that every iteration should be an experiment that allows you to learn how to make better products. Flow of code is not true progress, adding value to customers is, and if you aren't measuring UX in some way, you don't really know if you are just keeping busy or truly adding value.

          That's my view...


          On Feb 6, 2012, at 12:10 PM, kerrykimbrough wrote:

           



          Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've ever seen would use it. No one wants to fuss with this much analysis and data entry. And, if the team is using Rally or similar tracking tools, there is zero interest in data that's not in the tool.

          My 2nd reaction is that this matrix misses the point. What we need is good flow. We need work to arrive at each point in the flow completely ready for the next step. Devs don't want to fuss with your measures of UI design readiness. They want it to be ready.

          We already know the waterfall flow, with its big hopeless handoffs, doesn't work. Instead, we seek a more continous and incremental flow. But I don't see how the UXI matrix contributes.


        • Helen
          Jon, I am really interested in your UXI Matrix and appreciate the Excel template. I have been working on getting UX into Agile where I work, and I like the
          Message 4 of 15 , Feb 8, 2012
          • 0 Attachment
            Jon, I am really interested in your UXI Matrix and appreciate the
            Excel template.

            I have been working on getting UX into Agile where I work, and I like
            the suggestion of at least displaying the matrix in the Agile meeting
            room to at least generate discussion as to what a UX process really
            involves.


            I do have to agree with Peter on his statement where he disagrees with
            the premise that UX matters more for consumer products....it should be
            matter for everyone and everything :-) It definitely is a harder
            sell in an environment where you are dealing with enterprise
            applications and when you are procuring vendor applications. I have
            been advocating for UX evaluation when it comes to RFP vendor
            evaluation of enterprise applications and we are now embedded in our
            procurement process.

            Thanks again Jon.

            Helen

            On 2/7/12, Peter Gfader <peter@...> wrote:
            > Hi Jon
            >
            > Interesting idea.
            >
            >.
            >
            >
            >>>UX matters more for consumer products
            >
            > I dont agree. UX matters for everone and everything. It is just harder to
            > sell in software enterprises.
            > 2 selling points might be:
            > #1 Money savings in customer support (Less phone calls, less support staff).
            > #2 Customer satisfaction (harder to measure and sell... I agree)
            >
            >
            > In my experience, it is very valuable to have a UX guy on the Scrum
            > Team form the beginning. This way he ensures great UX (maybe by using the
            > UXI Matrix).
            >
            >
            > .peter.gfader. (current mood = happy!)
            > http://blog.gfader.com/
            >
            >
            >
            > On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough
            > <kerry@...>wrote:
            >
            >> **
            >>
            >>
            >>
            >>
            >> Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've
            >> ever seen would use it. No one wants to fuss with this much analysis and
            >> data entry. And, if the team is using Rally or similar tracking tools,
            >> there is zero interest in data that's not in the tool.
            >>
            >> My 2nd reaction is that this matrix misses the point. What we need is good
            >> flow. We need work to arrive at each point in the flow completely ready
            >> for
            >> the next step. Devs don't want to fuss with your measures of UI design
            >> readiness. They want it to be ready.
            >>
            >> We already know the waterfall flow, with its big hopeless handoffs,
            >> doesn't work. Instead, we seek a more continous and incremental flow. But
            >> I
            >> don't see how the UXI matrix contributes.
            >>
            >>
            >>
            >
            >
            >
            > --
            >
          • kerrykimbrough
            Sorry, there s a lot for me to clarify about my previous critique. I don t think the factors modeled in the UXI matrix are useless -- I think they are crucial.
            Message 5 of 15 , Feb 8, 2012
            • 0 Attachment
              Sorry, there's a lot for me to clarify about my previous critique.

              I don't think the factors modeled in the UXI matrix are useless -- I think they are crucial. I am not disinterested in UX metrics -- nor are most devs I've worked with. I agree that making UX work more visible is valuable for the whole team. I agree that it's pointless to deliver without accounting for the value added by the UI. Let us stipulate that none of those points are in dispute.

              The UXI matrix was offered as a way to help teams integrate UX into Scrum. I interpret that to mean "into the Scrum process practiced also by (non-UX) devs". But does it? That's where I have my doubts.

              How to connect UX with dev in an agile process? I have a POV, but I'm still trying to figure it out. Which is why I'm here.

              I see that many of the crucial UCD activities -- identifying personas, sifting alternative design visions, contextual research to validate, etc. -- must occur as a prelude to full-bore dev work. (Note that I'm glossing over earlier kinds of collaboration with devs, via "design in browser", etc., which I think is a great idea.) Until this work is sufficiently done, the story does not even exist -- in the *form needed by devs* to do their work. But when it is done, devs can proceed without understanding all of the details of what it took to bring the work to their input queue. Indeed, I'd say "..*must* proceed..." -- otherwise, we have a disruption in the flow. Please understand that, per my previous stipulation, I'm *not* saying "...must proceed without any concern for or regard for or interest in...". No, I'm saying something quite different.

              I don't see how the UXI matrix helps devs proceed with their work. I do see how it might help UX folks manage their prior work. But I'm less clear on how it helps to integrate UX into Scrum.

              To me, the UXI matrix looks a bit fussy, because I think simpler techniques could work better. Has any UX team tried using a Kanban board to model their work? The columns might capture much of the same content as the matrix. But a Kanban board would be easier to manage, would offer greater visibility, and would better help spot bottlenecks on a day-to-day basis.

              Regards,
              Kerry
            • Jon Innes
              Peter, We agree, mostly. I believe UX is becoming more important than ever, and awareness is only increasing. I was just pointing out that in 10 years of
              Message 6 of 15 , Feb 8, 2012
              • 0 Attachment
                Peter,

                We agree, mostly.

                I believe UX is becoming more important than ever, and awareness is only increasing. I was just pointing out that in 10 years of studying agile and talking to the early proponents of it, I realized the reason agile pioneers didn't highlight UX is that they were working in a context where there was no real concept of UX. Just read their accounts or talk to them like I did. Mike Cohn and Eric Ries are good example of agile proponents that do get UX.

                UX methods have largely been invented at product companies. Intuit, Apple, Xerox, IBM, DEC, and others did usability testing and hired UI designers in the 80's attempting to create products for mass consumer adoption. Now most enterprise software companies are attempting to do UX at the same level as consumer companies. I'm familiar with this, having been an early UX team member and UX manager/director at several of the most successful enterprise software companies.

                My comment was that UX matters more for consumer products, not that UX doesn't matter at all for enterprise products. Enterprise software is often 10x less usable than eCommerce sites. This is because of the feedback mechanisms inherent in enterprise software sales. Corporate users don't get to choose the software, and the cost of switching is bigger than clicking to another site. I've published articles and given talks on this. Your selling points listed below are correct, but the fact remains that UX has more influence on the success of consumer products and websites than it does on enterprise software or IT departmental budgets. Even Steve Jobs noted this once in an interview.

                I'm just trying to share my experience of being the UX guy on the scrum team, or cleaning up the mess that happens when the team lacks any UX skills. It's not enough to be there, you have to help the team understand that releasing software is just a step to meeting customer needs, but just releasing software that nobody wants or that is unusable isn't the fastest way to learn how to build great products.

                Jon

                On Feb 7, 2012, at 1:27 AM, Peter Gfader wrote:

                 

                Hi Jon


                Interesting idea. 

                My 2 cents...

                >>agile and UX methods evolved for different purposes, supporting different values. Agile methods were developed without consideration for UX best practices. Early agile pioneers were working on in-house IT projects (custom software) or enterprise software
                I think the close past and present shows us a different picture. A lot of software development companies embrace good UX from the beginning. And only those that do, are successful in the long run.


                >>UX matters more for consumer products
                I dont agree. UX matters for everone and everything. It is just harder to sell in software enterprises.
                2 selling points might be: 
                #1 Money savings in customer support (Less phone calls, less support staff).
                #2 Customer satisfaction (harder to measure and sell... I agree)


                In my experience, it is very valuable to have a UX guy on the Scrum Team form the beginning. This way he ensures great UX (maybe by using the UXI Matrix).


                  .peter.gfader. (current mood = happy!) 



                On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough <kerry@...> wrote:
                 



                Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've ever seen would use it. No one wants to fuss with this much analysis and data entry. And, if the team is using Rally or similar tracking tools, there is zero interest in data that's not in the tool.

                My 2nd reaction is that this matrix misses the point. What we need is good flow. We need work to arrive at each point in the flow completely ready for the next step. Devs don't want to fuss with your measures of UI design readiness. They want it to be ready.

                We already know the waterfall flow, with its big hopeless handoffs, doesn't work. Instead, we seek a more continous and incremental flow. But I don't see how the UXI matrix contributes.




                --




              • Jon Innes
                Tom, I m trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom).
                Message 7 of 15 , Feb 8, 2012
                • 0 Attachment
                  Tom,

                  I'm trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom). Tracking stuff by persona vs. story shows some interesting stuff. 

                  I was working on a big project as a consultant 2 years ago and we did this analysis and we identified a target persona, but in fact most of the stories were targeted at "other users."  Now that might be OK for a given sprint for some reason, but you can't keep focusing on users who aren't in your target persona over time. Something is wrong there. Same is true if you claim to be improving the UX and the UXI Matrix summary scores aren't moving in the right direction.

                  Your summary of what I'm tracking per story is accurate. Instead of tracking big waterfall delivery style, I'm tracking small UX tasks by story, just like agile proposes you do for dev & QA. I'm just mapping the UX work to the same stories the development team is using. I'd use the same tool as development if I could. I've used stickies on a wall (pure Scrum/Kanban style) and Excel, and even tried to do it in agile specific tools. The downside of things other than Excel have always been creating overall metrics easily. Just as many Scrum masters gravitate to Excel to track burn down if they use Scrum boards. 

                  Sounds like you are admitting to using Excel, just like Austin & I have :)

                  In terms of big picture, I've developed some dashboards that include UX metrics (% stories UX staff has designed, amount of user involvement, overall task completion rates, sat scores, etc.). This becomes easier if you have what's in the UXI Matrix example shown in the article. It's really hard if you don't have it.



                  On Feb 7, 2012, at 9:17 AM, Tom Hume wrote:

                   

                  It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.


                  I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

                  This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

                  Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

                  On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

                  So, here's the constructive part: what information would your perfect
                  product backlog track?




                  --
                  Future Platforms: hungry and foolish since 2000
                  work: Tom.Hume@... play: tomhume.org both: +447971781422



                • Adam Sroka
                  ... There are a few other names that should be on that list, not the least of whom is Jeff Patton. ... I agree with your analysis. That is the current state of
                  Message 8 of 15 , Feb 8, 2012
                  • 0 Attachment
                    On Wed, Feb 8, 2012 at 2:10 PM, Jon Innes <jinnes@...> wrote:
                    >
                    >
                    >
                    > Peter,
                    >
                    > We agree, mostly.
                    >
                    > I believe UX is becoming more important than ever, and awareness is only increasing. I was just pointing out that in 10 years of studying agile and talking to the early proponents of it, I realized the reason agile pioneers didn't highlight UX is that they were working in a context where there was no real concept of UX. Just read their accounts or talk to them like I did. Mike Cohn and Eric Ries are good example of agile proponents that do get UX.
                    >

                    There are a few other names that should be on that list, not the least
                    of whom is Jeff Patton.

                    > UX methods have largely been invented at product companies. Intuit, Apple, Xerox, IBM, DEC, and others did usability testing and hired UI designers in the 80's attempting to create products for mass consumer adoption. Now most enterprise software companies are attempting to do UX at the same level as consumer companies. I'm familiar with this, having been an early UX team member and UX manager/director at several of the most successful enterprise software companies.
                    >
                    > My comment was that UX matters more for consumer products, not that UX doesn't matter at all for enterprise products. Enterprise software is often 10x less usable than eCommerce sites. This is because of the feedback mechanisms inherent in enterprise software sales. Corporate users don't get to choose the software, and the cost of switching is bigger than clicking to another site. I've published articles and given talks on this. Your selling points listed below are correct, but the fact remains that UX has more influence on the success of consumer products and websites than it does on enterprise software or IT departmental budgets. Even Steve Jobs noted this once in an interview.
                    >

                    I agree with your analysis. That is the current state of things, but
                    perhaps we could do better...

                    I could imagine a scenario where having an internal customer for IT
                    projects would make it much easier and much faster to get feedback
                    about the quality and usability of your app. Presuming you could use
                    this information to improve the usability of your internal apps you
                    could also feed some of that information forward to the consumer side,
                    assuming that you work for one of the many organizations that does
                    both.

                    A good example of this is the Java IDE from JetBrains (IntelliJ IDEA.)
                    The folks who write it also use it. So, they are using the app to
                    develop the app and fixing usability issues that they encounter along
                    the way. They also get feedback from customers that they use to
                    improve it as well. You might not think IDEA is the best example of
                    great usability (Although IDEs are very complicated tools and getting
                    perfect usability out of them is not an easy problem,) but if you had
                    sufficient usability expertise combined with the kind of information
                    they are getting you could leverage it to do a lot.

                    Of course, there are all sorts of reasons why this information doesn't
                    get communicated or used in many organizations, but most of those
                    reasons aren't very good (Such as policies that discourage direct
                    communication between parts of an organization.)
                  • Jon Innes
                    You re welcome Helen, As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. If you start using
                    Message 9 of 15 , Feb 8, 2012
                    • 0 Attachment
                      You're welcome Helen,

                      As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. 

                      If you start using the standard method CIF proposes for vendor evaluation, you'll have some of the key metrics shown by story in the UXI Matrix. 

                      Just modify the UXI Matrix to add columns for task completion rates for multiple products. 

                      You can do the same for traditional A/B testing metrics like conversion rates and time on site.

                      Good luck with it and please let me know if you have any ideas for improving it.

                      Jon


                      On Feb 8, 2012, at 8:43 AM, Helen wrote:

                       

                      Jon, I am really interested in your UXI Matrix and appreciate the
                      Excel template.

                      I have been working on getting UX into Agile where I work, and I like
                      the suggestion of at least displaying the matrix in the Agile meeting
                      room to at least generate discussion as to what a UX process really
                      involves.

                      I do have to agree with Peter on his statement where he disagrees with
                      the premise that UX matters more for consumer products....it should be
                      matter for everyone and everything :-) It definitely is a harder
                      sell in an environment where you are dealing with enterprise
                      applications and when you are procuring vendor applications. I have
                      been advocating for UX evaluation when it comes to RFP vendor
                      evaluation of enterprise applications and we are now embedded in our
                      procurement process.

                      Thanks again Jon.

                      Helen

                      On 2/7/12, Peter Gfader <peter@...> wrote:
                      > Hi Jon
                      >
                      > Interesting idea.
                      >
                      >.
                      >
                      >
                      >>>UX matters more for consumer products
                      >
                      > I dont agree. UX matters for everone and everything. It is just harder to
                      > sell in software enterprises.
                      > 2 selling points might be:
                      > #1 Money savings in customer support (Less phone calls, less support staff).
                      > #2 Customer satisfaction (harder to measure and sell... I agree)
                      >
                      >
                      > In my experience, it is very valuable to have a UX guy on the Scrum
                      > Team form the beginning. This way he ensures great UX (maybe by using the
                      > UXI Matrix).
                      >
                      >
                      > .peter.gfader. (current mood = happy!)
                      > http://blog.gfader.com/
                      >
                      >
                      >
                      > On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough
                      > <kerry@...>wrote:
                      >
                      >> **
                      >>
                      >>
                      >>
                      >>
                      >> Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've
                      >> ever seen would use it. No one wants to fuss with this much analysis and
                      >> data entry. And, if the team is using Rally or similar tracking tools,
                      >> there is zero interest in data that's not in the tool.
                      >>
                      >> My 2nd reaction is that this matrix misses the point. What we need is good
                      >> flow. We need work to arrive at each point in the flow completely ready
                      >> for
                      >> the next step. Devs don't want to fuss with your measures of UI design
                      >> readiness. They want it to be ready.
                      >>
                      >> We already know the waterfall flow, with its big hopeless handoffs,
                      >> doesn't work. Instead, we seek a more continous and incremental flow. But
                      >> I
                      >> don't see how the UXI matrix contributes.
                      >>
                      >>
                      >>
                      >
                      >
                      >
                      > --
                      >


                    • Jon Innes
                      Just to clarify, I realized my last email might be misinterpreted... Another way to use the UXI Matrix for comparing two products/sites or design variations
                      Message 10 of 15 , Feb 8, 2012
                      • 0 Attachment
                        Just to clarify, I realized my last email might be misinterpreted...

                        Another way to use the UXI Matrix for comparing two products/sites or design variations (or even two sprints) is to make a copy of the UXI Matrix as a second sheet, then make a third sheet summarizing the differences. 

                        Doing comparisons is easy that way, either between two design variations you're testing using multivariate techniques or two products you're comparing that support a shared set of stories. 

                        I'd be curious if anyone can do this in an agile tracking tool like Jira, and how well that works...

                        PS congrats for getting embedded in the procurement process, you're now one of the rare companies that measures UX in IT!

                        On Feb 8, 2012, at 3:09 PM, Jon Innes wrote:

                        You're welcome Helen,

                        As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. 

                        If you start using the standard method CIF proposes for vendor evaluation, you'll have some of the key metrics shown by story in the UXI Matrix. 

                        Just modify the UXI Matrix to add columns for task completion rates for multiple products. 

                        You can do the same for traditional A/B testing metrics like conversion rates and time on site.

                        Good luck with it and please let me know if you have any ideas for improving it.

                        Jon


                        On Feb 8, 2012, at 8:43 AM, Helen wrote:

                         

                        Jon, I am really interested in your UXI Matrix and appreciate the
                        Excel template.

                        I have been working on getting UX into Agile where I work, and I like
                        the suggestion of at least displaying the matrix in the Agile meeting
                        room to at least generate discussion as to what a UX process really
                        involves.

                        I do have to agree with Peter on his statement where he disagrees with
                        the premise that UX matters more for consumer products....it should be
                        matter for everyone and everything :-) It definitely is a harder
                        sell in an environment where you are dealing with enterprise
                        applications and when you are procuring vendor applications. I have
                        been advocating for UX evaluation when it comes to RFP vendor
                        evaluation of enterprise applications and we are now embedded in our
                        procurement process.

                        Thanks again Jon.

                        Helen

                        On 2/7/12, Peter Gfader <peter@...> wrote:
                        > Hi Jon
                        >
                        > Interesting idea.
                        >
                        >.
                        >
                        >
                        >>>UX matters more for consumer products
                        >
                        > I dont agree. UX matters for everone and everything. It is just harder to
                        > sell in software enterprises.
                        > 2 selling points might be:
                        > #1 Money savings in customer support (Less phone calls, less support staff).
                        > #2 Customer satisfaction (harder to measure and sell... I agree)
                        >
                        >
                        > In my experience, it is very valuable to have a UX guy on the Scrum
                        > Team form the beginning. This way he ensures great UX (maybe by using the
                        > UXI Matrix).
                        >
                        >
                        > .peter.gfader. (current mood = happy!)
                        > http://blog.gfader.com/
                        >
                        >
                        >
                        > On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough
                        > <kerry@...>wrote:
                        >
                        >> **
                        >>
                        >>
                        >>
                        >>
                        >> Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've
                        >> ever seen would use it. No one wants to fuss with this much analysis and
                        >> data entry. And, if the team is using Rally or similar tracking tools,
                        >> there is zero interest in data that's not in the tool.
                        >>
                        >> My 2nd reaction is that this matrix misses the point. What we need is good
                        >> flow. We need work to arrive at each point in the flow completely ready
                        >> for
                        >> the next step. Devs don't want to fuss with your measures of UI design
                        >> readiness. They want it to be ready.
                        >>
                        >> We already know the waterfall flow, with its big hopeless handoffs,
                        >> doesn't work. Instead, we seek a more continous and incremental flow. But
                        >> I
                        >> don't see how the UXI matrix contributes.
                        >>
                        >>
                        >>
                        >
                        >
                        >
                        > --
                        >



                      • Tom Hume
                        Jon I agree that looking at how relevant work is to personas you re targeting is useful. I think there s lots of useful stuff to track in a project which I
                        Message 11 of 15 , Feb 9, 2012
                        • 0 Attachment
                          Jon

                          I agree that looking at how relevant work is to personas you're targeting is useful. I think there's lots of useful stuff to track in a project which I wouldn't naturally put onto a product backlog: acceptance tests, development tasks, design tasks, etc. Not having it on the backlog doesn't mean it isn't valuable. I've never found it necessary to track individual dev tasks on a product backlog, for instance. I like having a backlog that's simple and meaningful to everyone on the team, and unless the team were asking for it I wouldn't float details "to the top" like this. If I wanted UX metrics, I'd probably track them elsewhere (or ask the designer(s) to do that - it's probably a tool they could benefit from) to keep the main backlog clean.

                          If stories need to get some work done before they're ready, I /would/ track that, and would mark the state ("ready for development", say) in the backlog. I like using colours for this because it lets you see a historical record of state transitions, and easily spot patterns. 

                          This is not the same as suggesting that disciplines be silo'd or not collaborate - I've just tended to encourage that collaboration in other places.

                          Yes, I'll fess up - I use spreadsheets for this kind of thing :) Google Docs though (no worries about who has the latest version, visible in and our of the office, and hang around in one place forever). I've not found a tool that works better for me than them, but then I've not done an exhaustive search either (never felt the need).

                          YMMV of course (and it sounds like it does)

                          Tom

                          On 8 February 2012 22:47, Jon Innes <jinnes@...> wrote:
                          I'm trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom). Tracking stuff by persona vs. story shows some interesting stuff. 

                          I was working on a big project as a consultant 2 years ago and we did this analysis and we identified a target persona, but in fact most of the stories were targeted at "other users."  Now that might be OK for a given sprint for some reason, but you can't keep focusing on users who aren't in your target persona over time. Something is wrong there. Same is true if you claim to be improving the UX and the UXI Matrix summary scores aren't moving in the right direction.

                          Your summary of what I'm tracking per story is accurate. Instead of tracking big waterfall delivery style, I'm tracking small UX tasks by story, just like agile proposes you do for dev & QA. I'm just mapping the UX work to the same stories the development team is using. I'd use the same tool as development if I could. I've used stickies on a wall (pure Scrum/Kanban style) and Excel, and even tried to do it in agile specific tools. The downside of things other than Excel have always been creating overall metrics easily. Just as many Scrum masters gravitate to Excel to track burn down if they use Scrum boards. 

                          Sounds like you are admitting to using Excel, just like Austin & I have :)

                          In terms of big picture, I've developed some dashboards that include UX metrics (% stories UX staff has designed, amount of user involvement, overall task completion rates, sat scores, etc.). This becomes easier if you have what's in the UXI Matrix example shown in the article. It's really hard if you don't have it.



                          On Feb 7, 2012, at 9:17 AM, Tom Hume wrote:

                           

                          It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.


                          I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

                          This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

                          Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

                          On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

                          So, here's the constructive part: what information would your perfect
                          product backlog track?




                          --
                          Future Platforms: hungry and foolish since 2000
                          work: Tom.Hume@... play: tomhume.org both: +447971781422






                          --
                          Future Platforms: hungry and foolish since 2000
                          work: Tom.Hume@... play: tomhume.org both: +447971781422

                        • Peter Gfader
                          Hi Jon #1 Thanks for clarification #2 Good point regarding: Corporate users don t get to choose the software Someone else chooses and drives the software,
                          Message 12 of 15 , Feb 13, 2012
                          • 0 Attachment
                            Hi Jon

                            #1
                            Thanks for clarification

                            #2
                            Good point regarding: "Corporate users don't get to choose the software"
                            Someone else chooses and drives the software, but doesnt need to use it in the end...
                            A good way to overcome this,  might be Adam's suggestion with dogfooding: "internal customer for IT projects"


                            Good work!


                              .peter.gfader. (current mood = happy!) 



                            On Thu, Feb 9, 2012 at 11:04 AM, Tom Hume <Tom.Hume@...> wrote:
                             

                            Jon


                            I agree that looking at how relevant work is to personas you're targeting is useful. I think there's lots of useful stuff to track in a project which I wouldn't naturally put onto a product backlog: acceptance tests, development tasks, design tasks, etc. Not having it on the backlog doesn't mean it isn't valuable. I've never found it necessary to track individual dev tasks on a product backlog, for instance. I like having a backlog that's simple and meaningful to everyone on the team, and unless the team were asking for it I wouldn't float details "to the top" like this. If I wanted UX metrics, I'd probably track them elsewhere (or ask the designer(s) to do that - it's probably a tool they could benefit from) to keep the main backlog clean.

                            If stories need to get some work done before they're ready, I /would/ track that, and would mark the state ("ready for development", say) in the backlog. I like using colours for this because it lets you see a historical record of state transitions, and easily spot patterns. 

                            This is not the same as suggesting that disciplines be silo'd or not collaborate - I've just tended to encourage that collaboration in other places.

                            Yes, I'll fess up - I use spreadsheets for this kind of thing :) Google Docs though (no worries about who has the latest version, visible in and our of the office, and hang around in one place forever). I've not found a tool that works better for me than them, but then I've not done an exhaustive search either (never felt the need).

                            YMMV of course (and it sounds like it does)

                            Tom

                            On 8 February 2012 22:47, Jon Innes <jinnes@...> wrote:
                            I'm trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom). Tracking stuff by persona vs. story shows some interesting stuff. 

                            I was working on a big project as a consultant 2 years ago and we did this analysis and we identified a target persona, but in fact most of the stories were targeted at "other users."  Now that might be OK for a given sprint for some reason, but you can't keep focusing on users who aren't in your target persona over time. Something is wrong there. Same is true if you claim to be improving the UX and the UXI Matrix summary scores aren't moving in the right direction.

                            Your summary of what I'm tracking per story is accurate. Instead of tracking big waterfall delivery style, I'm tracking small UX tasks by story, just like agile proposes you do for dev & QA. I'm just mapping the UX work to the same stories the development team is using. I'd use the same tool as development if I could. I've used stickies on a wall (pure Scrum/Kanban style) and Excel, and even tried to do it in agile specific tools. The downside of things other than Excel have always been creating overall metrics easily. Just as many Scrum masters gravitate to Excel to track burn down if they use Scrum boards. 

                            Sounds like you are admitting to using Excel, just like Austin & I have :)

                            In terms of big picture, I've developed some dashboards that include UX metrics (% stories UX staff has designed, amount of user involvement, overall task completion rates, sat scores, etc.). This becomes easier if you have what's in the UXI Matrix example shown in the article. It's really hard if you don't have it.



                            On Feb 7, 2012, at 9:17 AM, Tom Hume wrote:

                             

                            It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.


                            I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

                            This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

                            Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

                            On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

                            So, here's the constructive part: what information would your perfect
                            product backlog track?




                            --
                            Future Platforms: hungry and foolish since 2000
                            work: Tom.Hume@... play: tomhume.org both: +447971781422






                            --
                            Future Platforms: hungry and foolish since 2000
                            work: Tom.Hume@... play: tomhume.org both: +447971781422




                            --

                          Your message has been successfully submitted and would be delivered to recipients shortly.