Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Re: UXI Matrix

Expand Messages
  • Adam Sroka
    ... There are a few other names that should be on that list, not the least of whom is Jeff Patton. ... I agree with your analysis. That is the current state of
    Message 1 of 15 , Feb 8, 2012
    • 0 Attachment
      On Wed, Feb 8, 2012 at 2:10 PM, Jon Innes <jinnes@...> wrote:
      >
      >
      >
      > Peter,
      >
      > We agree, mostly.
      >
      > I believe UX is becoming more important than ever, and awareness is only increasing. I was just pointing out that in 10 years of studying agile and talking to the early proponents of it, I realized the reason agile pioneers didn't highlight UX is that they were working in a context where there was no real concept of UX. Just read their accounts or talk to them like I did. Mike Cohn and Eric Ries are good example of agile proponents that do get UX.
      >

      There are a few other names that should be on that list, not the least
      of whom is Jeff Patton.

      > UX methods have largely been invented at product companies. Intuit, Apple, Xerox, IBM, DEC, and others did usability testing and hired UI designers in the 80's attempting to create products for mass consumer adoption. Now most enterprise software companies are attempting to do UX at the same level as consumer companies. I'm familiar with this, having been an early UX team member and UX manager/director at several of the most successful enterprise software companies.
      >
      > My comment was that UX matters more for consumer products, not that UX doesn't matter at all for enterprise products. Enterprise software is often 10x less usable than eCommerce sites. This is because of the feedback mechanisms inherent in enterprise software sales. Corporate users don't get to choose the software, and the cost of switching is bigger than clicking to another site. I've published articles and given talks on this. Your selling points listed below are correct, but the fact remains that UX has more influence on the success of consumer products and websites than it does on enterprise software or IT departmental budgets. Even Steve Jobs noted this once in an interview.
      >

      I agree with your analysis. That is the current state of things, but
      perhaps we could do better...

      I could imagine a scenario where having an internal customer for IT
      projects would make it much easier and much faster to get feedback
      about the quality and usability of your app. Presuming you could use
      this information to improve the usability of your internal apps you
      could also feed some of that information forward to the consumer side,
      assuming that you work for one of the many organizations that does
      both.

      A good example of this is the Java IDE from JetBrains (IntelliJ IDEA.)
      The folks who write it also use it. So, they are using the app to
      develop the app and fixing usability issues that they encounter along
      the way. They also get feedback from customers that they use to
      improve it as well. You might not think IDEA is the best example of
      great usability (Although IDEs are very complicated tools and getting
      perfect usability out of them is not an easy problem,) but if you had
      sufficient usability expertise combined with the kind of information
      they are getting you could leverage it to do a lot.

      Of course, there are all sorts of reasons why this information doesn't
      get communicated or used in many organizations, but most of those
      reasons aren't very good (Such as policies that discourage direct
      communication between parts of an organization.)
    • Jon Innes
      You re welcome Helen, As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. If you start using
      Message 2 of 15 , Feb 8, 2012
      • 0 Attachment
        You're welcome Helen,

        As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. 

        If you start using the standard method CIF proposes for vendor evaluation, you'll have some of the key metrics shown by story in the UXI Matrix. 

        Just modify the UXI Matrix to add columns for task completion rates for multiple products. 

        You can do the same for traditional A/B testing metrics like conversion rates and time on site.

        Good luck with it and please let me know if you have any ideas for improving it.

        Jon


        On Feb 8, 2012, at 8:43 AM, Helen wrote:

         

        Jon, I am really interested in your UXI Matrix and appreciate the
        Excel template.

        I have been working on getting UX into Agile where I work, and I like
        the suggestion of at least displaying the matrix in the Agile meeting
        room to at least generate discussion as to what a UX process really
        involves.

        I do have to agree with Peter on his statement where he disagrees with
        the premise that UX matters more for consumer products....it should be
        matter for everyone and everything :-) It definitely is a harder
        sell in an environment where you are dealing with enterprise
        applications and when you are procuring vendor applications. I have
        been advocating for UX evaluation when it comes to RFP vendor
        evaluation of enterprise applications and we are now embedded in our
        procurement process.

        Thanks again Jon.

        Helen

        On 2/7/12, Peter Gfader <peter@...> wrote:
        > Hi Jon
        >
        > Interesting idea.
        >
        >.
        >
        >
        >>>UX matters more for consumer products
        >
        > I dont agree. UX matters for everone and everything. It is just harder to
        > sell in software enterprises.
        > 2 selling points might be:
        > #1 Money savings in customer support (Less phone calls, less support staff).
        > #2 Customer satisfaction (harder to measure and sell... I agree)
        >
        >
        > In my experience, it is very valuable to have a UX guy on the Scrum
        > Team form the beginning. This way he ensures great UX (maybe by using the
        > UXI Matrix).
        >
        >
        > .peter.gfader. (current mood = happy!)
        > http://blog.gfader.com/
        >
        >
        >
        > On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough
        > <kerry@...>wrote:
        >
        >> **
        >>
        >>
        >>
        >>
        >> Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've
        >> ever seen would use it. No one wants to fuss with this much analysis and
        >> data entry. And, if the team is using Rally or similar tracking tools,
        >> there is zero interest in data that's not in the tool.
        >>
        >> My 2nd reaction is that this matrix misses the point. What we need is good
        >> flow. We need work to arrive at each point in the flow completely ready
        >> for
        >> the next step. Devs don't want to fuss with your measures of UI design
        >> readiness. They want it to be ready.
        >>
        >> We already know the waterfall flow, with its big hopeless handoffs,
        >> doesn't work. Instead, we seek a more continous and incremental flow. But
        >> I
        >> don't see how the UXI matrix contributes.
        >>
        >>
        >>
        >
        >
        >
        > --
        >


      • Jon Innes
        Just to clarify, I realized my last email might be misinterpreted... Another way to use the UXI Matrix for comparing two products/sites or design variations
        Message 3 of 15 , Feb 8, 2012
        • 0 Attachment
          Just to clarify, I realized my last email might be misinterpreted...

          Another way to use the UXI Matrix for comparing two products/sites or design variations (or even two sprints) is to make a copy of the UXI Matrix as a second sheet, then make a third sheet summarizing the differences. 

          Doing comparisons is easy that way, either between two design variations you're testing using multivariate techniques or two products you're comparing that support a shared set of stories. 

          I'd be curious if anyone can do this in an agile tracking tool like Jira, and how well that works...

          PS congrats for getting embedded in the procurement process, you're now one of the rare companies that measures UX in IT!

          On Feb 8, 2012, at 3:09 PM, Jon Innes wrote:

          You're welcome Helen,

          As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. 

          If you start using the standard method CIF proposes for vendor evaluation, you'll have some of the key metrics shown by story in the UXI Matrix. 

          Just modify the UXI Matrix to add columns for task completion rates for multiple products. 

          You can do the same for traditional A/B testing metrics like conversion rates and time on site.

          Good luck with it and please let me know if you have any ideas for improving it.

          Jon


          On Feb 8, 2012, at 8:43 AM, Helen wrote:

           

          Jon, I am really interested in your UXI Matrix and appreciate the
          Excel template.

          I have been working on getting UX into Agile where I work, and I like
          the suggestion of at least displaying the matrix in the Agile meeting
          room to at least generate discussion as to what a UX process really
          involves.

          I do have to agree with Peter on his statement where he disagrees with
          the premise that UX matters more for consumer products....it should be
          matter for everyone and everything :-) It definitely is a harder
          sell in an environment where you are dealing with enterprise
          applications and when you are procuring vendor applications. I have
          been advocating for UX evaluation when it comes to RFP vendor
          evaluation of enterprise applications and we are now embedded in our
          procurement process.

          Thanks again Jon.

          Helen

          On 2/7/12, Peter Gfader <peter@...> wrote:
          > Hi Jon
          >
          > Interesting idea.
          >
          >.
          >
          >
          >>>UX matters more for consumer products
          >
          > I dont agree. UX matters for everone and everything. It is just harder to
          > sell in software enterprises.
          > 2 selling points might be:
          > #1 Money savings in customer support (Less phone calls, less support staff).
          > #2 Customer satisfaction (harder to measure and sell... I agree)
          >
          >
          > In my experience, it is very valuable to have a UX guy on the Scrum
          > Team form the beginning. This way he ensures great UX (maybe by using the
          > UXI Matrix).
          >
          >
          > .peter.gfader. (current mood = happy!)
          > http://blog.gfader.com/
          >
          >
          >
          > On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough
          > <kerry@...>wrote:
          >
          >> **
          >>
          >>
          >>
          >>
          >> Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've
          >> ever seen would use it. No one wants to fuss with this much analysis and
          >> data entry. And, if the team is using Rally or similar tracking tools,
          >> there is zero interest in data that's not in the tool.
          >>
          >> My 2nd reaction is that this matrix misses the point. What we need is good
          >> flow. We need work to arrive at each point in the flow completely ready
          >> for
          >> the next step. Devs don't want to fuss with your measures of UI design
          >> readiness. They want it to be ready.
          >>
          >> We already know the waterfall flow, with its big hopeless handoffs,
          >> doesn't work. Instead, we seek a more continous and incremental flow. But
          >> I
          >> don't see how the UXI matrix contributes.
          >>
          >>
          >>
          >
          >
          >
          > --
          >



        • Tom Hume
          Jon I agree that looking at how relevant work is to personas you re targeting is useful. I think there s lots of useful stuff to track in a project which I
          Message 4 of 15 , Feb 9, 2012
          • 0 Attachment
            Jon

            I agree that looking at how relevant work is to personas you're targeting is useful. I think there's lots of useful stuff to track in a project which I wouldn't naturally put onto a product backlog: acceptance tests, development tasks, design tasks, etc. Not having it on the backlog doesn't mean it isn't valuable. I've never found it necessary to track individual dev tasks on a product backlog, for instance. I like having a backlog that's simple and meaningful to everyone on the team, and unless the team were asking for it I wouldn't float details "to the top" like this. If I wanted UX metrics, I'd probably track them elsewhere (or ask the designer(s) to do that - it's probably a tool they could benefit from) to keep the main backlog clean.

            If stories need to get some work done before they're ready, I /would/ track that, and would mark the state ("ready for development", say) in the backlog. I like using colours for this because it lets you see a historical record of state transitions, and easily spot patterns. 

            This is not the same as suggesting that disciplines be silo'd or not collaborate - I've just tended to encourage that collaboration in other places.

            Yes, I'll fess up - I use spreadsheets for this kind of thing :) Google Docs though (no worries about who has the latest version, visible in and our of the office, and hang around in one place forever). I've not found a tool that works better for me than them, but then I've not done an exhaustive search either (never felt the need).

            YMMV of course (and it sounds like it does)

            Tom

            On 8 February 2012 22:47, Jon Innes <jinnes@...> wrote:
            I'm trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom). Tracking stuff by persona vs. story shows some interesting stuff. 

            I was working on a big project as a consultant 2 years ago and we did this analysis and we identified a target persona, but in fact most of the stories were targeted at "other users."  Now that might be OK for a given sprint for some reason, but you can't keep focusing on users who aren't in your target persona over time. Something is wrong there. Same is true if you claim to be improving the UX and the UXI Matrix summary scores aren't moving in the right direction.

            Your summary of what I'm tracking per story is accurate. Instead of tracking big waterfall delivery style, I'm tracking small UX tasks by story, just like agile proposes you do for dev & QA. I'm just mapping the UX work to the same stories the development team is using. I'd use the same tool as development if I could. I've used stickies on a wall (pure Scrum/Kanban style) and Excel, and even tried to do it in agile specific tools. The downside of things other than Excel have always been creating overall metrics easily. Just as many Scrum masters gravitate to Excel to track burn down if they use Scrum boards. 

            Sounds like you are admitting to using Excel, just like Austin & I have :)

            In terms of big picture, I've developed some dashboards that include UX metrics (% stories UX staff has designed, amount of user involvement, overall task completion rates, sat scores, etc.). This becomes easier if you have what's in the UXI Matrix example shown in the article. It's really hard if you don't have it.



            On Feb 7, 2012, at 9:17 AM, Tom Hume wrote:

             

            It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.


            I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

            This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

            Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

            On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

            So, here's the constructive part: what information would your perfect
            product backlog track?




            --
            Future Platforms: hungry and foolish since 2000
            work: Tom.Hume@... play: tomhume.org both: +447971781422






            --
            Future Platforms: hungry and foolish since 2000
            work: Tom.Hume@... play: tomhume.org both: +447971781422

          • Peter Gfader
            Hi Jon #1 Thanks for clarification #2 Good point regarding: Corporate users don t get to choose the software Someone else chooses and drives the software,
            Message 5 of 15 , Feb 13, 2012
            • 0 Attachment
              Hi Jon

              #1
              Thanks for clarification

              #2
              Good point regarding: "Corporate users don't get to choose the software"
              Someone else chooses and drives the software, but doesnt need to use it in the end...
              A good way to overcome this,  might be Adam's suggestion with dogfooding: "internal customer for IT projects"


              Good work!


                .peter.gfader. (current mood = happy!) 



              On Thu, Feb 9, 2012 at 11:04 AM, Tom Hume <Tom.Hume@...> wrote:
               

              Jon


              I agree that looking at how relevant work is to personas you're targeting is useful. I think there's lots of useful stuff to track in a project which I wouldn't naturally put onto a product backlog: acceptance tests, development tasks, design tasks, etc. Not having it on the backlog doesn't mean it isn't valuable. I've never found it necessary to track individual dev tasks on a product backlog, for instance. I like having a backlog that's simple and meaningful to everyone on the team, and unless the team were asking for it I wouldn't float details "to the top" like this. If I wanted UX metrics, I'd probably track them elsewhere (or ask the designer(s) to do that - it's probably a tool they could benefit from) to keep the main backlog clean.

              If stories need to get some work done before they're ready, I /would/ track that, and would mark the state ("ready for development", say) in the backlog. I like using colours for this because it lets you see a historical record of state transitions, and easily spot patterns. 

              This is not the same as suggesting that disciplines be silo'd or not collaborate - I've just tended to encourage that collaboration in other places.

              Yes, I'll fess up - I use spreadsheets for this kind of thing :) Google Docs though (no worries about who has the latest version, visible in and our of the office, and hang around in one place forever). I've not found a tool that works better for me than them, but then I've not done an exhaustive search either (never felt the need).

              YMMV of course (and it sounds like it does)

              Tom

              On 8 February 2012 22:47, Jon Innes <jinnes@...> wrote:
              I'm trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom). Tracking stuff by persona vs. story shows some interesting stuff. 

              I was working on a big project as a consultant 2 years ago and we did this analysis and we identified a target persona, but in fact most of the stories were targeted at "other users."  Now that might be OK for a given sprint for some reason, but you can't keep focusing on users who aren't in your target persona over time. Something is wrong there. Same is true if you claim to be improving the UX and the UXI Matrix summary scores aren't moving in the right direction.

              Your summary of what I'm tracking per story is accurate. Instead of tracking big waterfall delivery style, I'm tracking small UX tasks by story, just like agile proposes you do for dev & QA. I'm just mapping the UX work to the same stories the development team is using. I'd use the same tool as development if I could. I've used stickies on a wall (pure Scrum/Kanban style) and Excel, and even tried to do it in agile specific tools. The downside of things other than Excel have always been creating overall metrics easily. Just as many Scrum masters gravitate to Excel to track burn down if they use Scrum boards. 

              Sounds like you are admitting to using Excel, just like Austin & I have :)

              In terms of big picture, I've developed some dashboards that include UX metrics (% stories UX staff has designed, amount of user involvement, overall task completion rates, sat scores, etc.). This becomes easier if you have what's in the UXI Matrix example shown in the article. It's really hard if you don't have it.



              On Feb 7, 2012, at 9:17 AM, Tom Hume wrote:

               

              It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.


              I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

              This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

              Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

              On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

              So, here's the constructive part: what information would your perfect
              product backlog track?




              --
              Future Platforms: hungry and foolish since 2000
              work: Tom.Hume@... play: tomhume.org both: +447971781422






              --
              Future Platforms: hungry and foolish since 2000
              work: Tom.Hume@... play: tomhume.org both: +447971781422




              --

            Your message has been successfully submitted and would be delivered to recipients shortly.