Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Re: UXI Matrix

Expand Messages
  • Jon Innes
    You re welcome Helen, As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. If you start using
    Message 1 of 15 , Feb 8, 2012
    • 0 Attachment
      You're welcome Helen,

      As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. 

      If you start using the standard method CIF proposes for vendor evaluation, you'll have some of the key metrics shown by story in the UXI Matrix. 

      Just modify the UXI Matrix to add columns for task completion rates for multiple products. 

      You can do the same for traditional A/B testing metrics like conversion rates and time on site.

      Good luck with it and please let me know if you have any ideas for improving it.

      Jon


      On Feb 8, 2012, at 8:43 AM, Helen wrote:

       

      Jon, I am really interested in your UXI Matrix and appreciate the
      Excel template.

      I have been working on getting UX into Agile where I work, and I like
      the suggestion of at least displaying the matrix in the Agile meeting
      room to at least generate discussion as to what a UX process really
      involves.

      I do have to agree with Peter on his statement where he disagrees with
      the premise that UX matters more for consumer products....it should be
      matter for everyone and everything :-) It definitely is a harder
      sell in an environment where you are dealing with enterprise
      applications and when you are procuring vendor applications. I have
      been advocating for UX evaluation when it comes to RFP vendor
      evaluation of enterprise applications and we are now embedded in our
      procurement process.

      Thanks again Jon.

      Helen

      On 2/7/12, Peter Gfader <peter@...> wrote:
      > Hi Jon
      >
      > Interesting idea.
      >
      >.
      >
      >
      >>>UX matters more for consumer products
      >
      > I dont agree. UX matters for everone and everything. It is just harder to
      > sell in software enterprises.
      > 2 selling points might be:
      > #1 Money savings in customer support (Less phone calls, less support staff).
      > #2 Customer satisfaction (harder to measure and sell... I agree)
      >
      >
      > In my experience, it is very valuable to have a UX guy on the Scrum
      > Team form the beginning. This way he ensures great UX (maybe by using the
      > UXI Matrix).
      >
      >
      > .peter.gfader. (current mood = happy!)
      > http://blog.gfader.com/
      >
      >
      >
      > On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough
      > <kerry@...>wrote:
      >
      >> **
      >>
      >>
      >>
      >>
      >> Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've
      >> ever seen would use it. No one wants to fuss with this much analysis and
      >> data entry. And, if the team is using Rally or similar tracking tools,
      >> there is zero interest in data that's not in the tool.
      >>
      >> My 2nd reaction is that this matrix misses the point. What we need is good
      >> flow. We need work to arrive at each point in the flow completely ready
      >> for
      >> the next step. Devs don't want to fuss with your measures of UI design
      >> readiness. They want it to be ready.
      >>
      >> We already know the waterfall flow, with its big hopeless handoffs,
      >> doesn't work. Instead, we seek a more continous and incremental flow. But
      >> I
      >> don't see how the UXI matrix contributes.
      >>
      >>
      >>
      >
      >
      >
      > --
      >


    • Jon Innes
      Just to clarify, I realized my last email might be misinterpreted... Another way to use the UXI Matrix for comparing two products/sites or design variations
      Message 2 of 15 , Feb 8, 2012
      • 0 Attachment
        Just to clarify, I realized my last email might be misinterpreted...

        Another way to use the UXI Matrix for comparing two products/sites or design variations (or even two sprints) is to make a copy of the UXI Matrix as a second sheet, then make a third sheet summarizing the differences. 

        Doing comparisons is easy that way, either between two design variations you're testing using multivariate techniques or two products you're comparing that support a shared set of stories. 

        I'd be curious if anyone can do this in an agile tracking tool like Jira, and how well that works...

        PS congrats for getting embedded in the procurement process, you're now one of the rare companies that measures UX in IT!

        On Feb 8, 2012, at 3:09 PM, Jon Innes wrote:

        You're welcome Helen,

        As for UX evaluations for vendor apps, part of my inspiration from this comes from being a part of the CIF project. 

        If you start using the standard method CIF proposes for vendor evaluation, you'll have some of the key metrics shown by story in the UXI Matrix. 

        Just modify the UXI Matrix to add columns for task completion rates for multiple products. 

        You can do the same for traditional A/B testing metrics like conversion rates and time on site.

        Good luck with it and please let me know if you have any ideas for improving it.

        Jon


        On Feb 8, 2012, at 8:43 AM, Helen wrote:

         

        Jon, I am really interested in your UXI Matrix and appreciate the
        Excel template.

        I have been working on getting UX into Agile where I work, and I like
        the suggestion of at least displaying the matrix in the Agile meeting
        room to at least generate discussion as to what a UX process really
        involves.

        I do have to agree with Peter on his statement where he disagrees with
        the premise that UX matters more for consumer products....it should be
        matter for everyone and everything :-) It definitely is a harder
        sell in an environment where you are dealing with enterprise
        applications and when you are procuring vendor applications. I have
        been advocating for UX evaluation when it comes to RFP vendor
        evaluation of enterprise applications and we are now embedded in our
        procurement process.

        Thanks again Jon.

        Helen

        On 2/7/12, Peter Gfader <peter@...> wrote:
        > Hi Jon
        >
        > Interesting idea.
        >
        >.
        >
        >
        >>>UX matters more for consumer products
        >
        > I dont agree. UX matters for everone and everything. It is just harder to
        > sell in software enterprises.
        > 2 selling points might be:
        > #1 Money savings in customer support (Less phone calls, less support staff).
        > #2 Customer satisfaction (harder to measure and sell... I agree)
        >
        >
        > In my experience, it is very valuable to have a UX guy on the Scrum
        > Team form the beginning. This way he ensures great UX (maybe by using the
        > UXI Matrix).
        >
        >
        > .peter.gfader. (current mood = happy!)
        > http://blog.gfader.com/
        >
        >
        >
        > On Mon, Feb 6, 2012 at 9:10 PM, kerrykimbrough
        > <kerry@...>wrote:
        >
        >> **
        >>
        >>
        >>
        >>
        >> Jon, my 1st reaction to your UXI matrix concept is that no Scrum team I've
        >> ever seen would use it. No one wants to fuss with this much analysis and
        >> data entry. And, if the team is using Rally or similar tracking tools,
        >> there is zero interest in data that's not in the tool.
        >>
        >> My 2nd reaction is that this matrix misses the point. What we need is good
        >> flow. We need work to arrive at each point in the flow completely ready
        >> for
        >> the next step. Devs don't want to fuss with your measures of UI design
        >> readiness. They want it to be ready.
        >>
        >> We already know the waterfall flow, with its big hopeless handoffs,
        >> doesn't work. Instead, we seek a more continous and incremental flow. But
        >> I
        >> don't see how the UXI matrix contributes.
        >>
        >>
        >>
        >
        >
        >
        > --
        >



      • Tom Hume
        Jon I agree that looking at how relevant work is to personas you re targeting is useful. I think there s lots of useful stuff to track in a project which I
        Message 3 of 15 , Feb 9, 2012
        • 0 Attachment
          Jon

          I agree that looking at how relevant work is to personas you're targeting is useful. I think there's lots of useful stuff to track in a project which I wouldn't naturally put onto a product backlog: acceptance tests, development tasks, design tasks, etc. Not having it on the backlog doesn't mean it isn't valuable. I've never found it necessary to track individual dev tasks on a product backlog, for instance. I like having a backlog that's simple and meaningful to everyone on the team, and unless the team were asking for it I wouldn't float details "to the top" like this. If I wanted UX metrics, I'd probably track them elsewhere (or ask the designer(s) to do that - it's probably a tool they could benefit from) to keep the main backlog clean.

          If stories need to get some work done before they're ready, I /would/ track that, and would mark the state ("ready for development", say) in the backlog. I like using colours for this because it lets you see a historical record of state transitions, and easily spot patterns. 

          This is not the same as suggesting that disciplines be silo'd or not collaborate - I've just tended to encourage that collaboration in other places.

          Yes, I'll fess up - I use spreadsheets for this kind of thing :) Google Docs though (no worries about who has the latest version, visible in and our of the office, and hang around in one place forever). I've not found a tool that works better for me than them, but then I've not done an exhaustive search either (never felt the need).

          YMMV of course (and it sounds like it does)

          Tom

          On 8 February 2012 22:47, Jon Innes <jinnes@...> wrote:
          I'm trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom). Tracking stuff by persona vs. story shows some interesting stuff. 

          I was working on a big project as a consultant 2 years ago and we did this analysis and we identified a target persona, but in fact most of the stories were targeted at "other users."  Now that might be OK for a given sprint for some reason, but you can't keep focusing on users who aren't in your target persona over time. Something is wrong there. Same is true if you claim to be improving the UX and the UXI Matrix summary scores aren't moving in the right direction.

          Your summary of what I'm tracking per story is accurate. Instead of tracking big waterfall delivery style, I'm tracking small UX tasks by story, just like agile proposes you do for dev & QA. I'm just mapping the UX work to the same stories the development team is using. I'd use the same tool as development if I could. I've used stickies on a wall (pure Scrum/Kanban style) and Excel, and even tried to do it in agile specific tools. The downside of things other than Excel have always been creating overall metrics easily. Just as many Scrum masters gravitate to Excel to track burn down if they use Scrum boards. 

          Sounds like you are admitting to using Excel, just like Austin & I have :)

          In terms of big picture, I've developed some dashboards that include UX metrics (% stories UX staff has designed, amount of user involvement, overall task completion rates, sat scores, etc.). This becomes easier if you have what's in the UXI Matrix example shown in the article. It's really hard if you don't have it.



          On Feb 7, 2012, at 9:17 AM, Tom Hume wrote:

           

          It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.


          I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

          This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

          Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

          On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

          So, here's the constructive part: what information would your perfect
          product backlog track?




          --
          Future Platforms: hungry and foolish since 2000
          work: Tom.Hume@... play: tomhume.org both: +447971781422






          --
          Future Platforms: hungry and foolish since 2000
          work: Tom.Hume@... play: tomhume.org both: +447971781422

        • Peter Gfader
          Hi Jon #1 Thanks for clarification #2 Good point regarding: Corporate users don t get to choose the software Someone else chooses and drives the software,
          Message 4 of 15 , Feb 13, 2012
          • 0 Attachment
            Hi Jon

            #1
            Thanks for clarification

            #2
            Good point regarding: "Corporate users don't get to choose the software"
            Someone else chooses and drives the software, but doesnt need to use it in the end...
            A good way to overcome this,  might be Adam's suggestion with dogfooding: "internal customer for IT projects"


            Good work!


              .peter.gfader. (current mood = happy!) 



            On Thu, Feb 9, 2012 at 11:04 AM, Tom Hume <Tom.Hume@...> wrote:
             

            Jon


            I agree that looking at how relevant work is to personas you're targeting is useful. I think there's lots of useful stuff to track in a project which I wouldn't naturally put onto a product backlog: acceptance tests, development tasks, design tasks, etc. Not having it on the backlog doesn't mean it isn't valuable. I've never found it necessary to track individual dev tasks on a product backlog, for instance. I like having a backlog that's simple and meaningful to everyone on the team, and unless the team were asking for it I wouldn't float details "to the top" like this. If I wanted UX metrics, I'd probably track them elsewhere (or ask the designer(s) to do that - it's probably a tool they could benefit from) to keep the main backlog clean.

            If stories need to get some work done before they're ready, I /would/ track that, and would mark the state ("ready for development", say) in the backlog. I like using colours for this because it lets you see a historical record of state transitions, and easily spot patterns. 

            This is not the same as suggesting that disciplines be silo'd or not collaborate - I've just tended to encourage that collaboration in other places.

            Yes, I'll fess up - I use spreadsheets for this kind of thing :) Google Docs though (no worries about who has the latest version, visible in and our of the office, and hang around in one place forever). I've not found a tool that works better for me than them, but then I've not done an exhaustive search either (never felt the need).

            YMMV of course (and it sounds like it does)

            Tom

            On 8 February 2012 22:47, Jon Innes <jinnes@...> wrote:
            I'm trying to do two things. Track the state of individual items (stories on the rows) and track the big picture (see summary rows at the bottom). Tracking stuff by persona vs. story shows some interesting stuff. 

            I was working on a big project as a consultant 2 years ago and we did this analysis and we identified a target persona, but in fact most of the stories were targeted at "other users."  Now that might be OK for a given sprint for some reason, but you can't keep focusing on users who aren't in your target persona over time. Something is wrong there. Same is true if you claim to be improving the UX and the UXI Matrix summary scores aren't moving in the right direction.

            Your summary of what I'm tracking per story is accurate. Instead of tracking big waterfall delivery style, I'm tracking small UX tasks by story, just like agile proposes you do for dev & QA. I'm just mapping the UX work to the same stories the development team is using. I'd use the same tool as development if I could. I've used stickies on a wall (pure Scrum/Kanban style) and Excel, and even tried to do it in agile specific tools. The downside of things other than Excel have always been creating overall metrics easily. Just as many Scrum masters gravitate to Excel to track burn down if they use Scrum boards. 

            Sounds like you are admitting to using Excel, just like Austin & I have :)

            In terms of big picture, I've developed some dashboards that include UX metrics (% stories UX staff has designed, amount of user involvement, overall task completion rates, sat scores, etc.). This becomes easier if you have what's in the UXI Matrix example shown in the article. It's really hard if you don't have it.



            On Feb 7, 2012, at 9:17 AM, Tom Hume wrote:

             

            It strikes me that what you're tracking here is the state of individual backlog items. In your case, it's whether they're ready to move to development by virtue of the design work being completed. Further down the line, you might want to track whether a given backlog item is through development and ready for testing; prior to design you might want to track whether a given backlog item has been approved for design work to start.


            I've used colour-coding of cells in a backlog for this sort of thing; in a product backlog we would apply to the cell whose row indicates the backlog item and column indicates the sprint number. For a sprint backlog, replace the latter by the column indicating the day in the sprint.

            This makes it very clear what the state and/or readiness of current items is, maps neatly onto columns on a physical board, and lets you see patterns (either in-sprint or across them) of movement of backlog items between various states.

            Tracking every aspect of what makes a given backlog item ready to proceed into development might be appropriate, or might not - YMMV. One thing I like about backlogs is that they give us a useful summary of a project, as the expense of including every detail.

            On 7 February 2012 16:38, Austin Govella <austin.govella@...> wrote:

            So, here's the constructive part: what information would your perfect
            product backlog track?




            --
            Future Platforms: hungry and foolish since 2000
            work: Tom.Hume@... play: tomhume.org both: +447971781422






            --
            Future Platforms: hungry and foolish since 2000
            work: Tom.Hume@... play: tomhume.org both: +447971781422




            --

          Your message has been successfully submitted and would be delivered to recipients shortly.