Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-usability] Serendipity on anthrodesign mailing list

Expand Messages
  • Jon Innes
    I m here reading this having a beer after spending 14 hours today helping a company try to do this stuff right. The problem with the approaches described below
    Message 1 of 15 , Jun 19, 2013
    • 0 Attachment
      I'm here reading this having a beer after spending 14 hours today helping a company try to do this stuff right.

      The problem with the approaches described below is that some people who do research (who are not really UX people) don't know how to do UX. UX requires design focused research. 

      These folks have titles like UER. This is because they don't think of UE/UX as including research AND design a condition of being UX by definition. These are researchers that throw their results over the wall like it has value on it's own. It doesn't. They don't work with designers, or for that matter the rest of the team, to create the solution.

      Research in the absence of design considerations has no value in a product oriented company. If you want to do that type of work go apply for an academic position or a government grant. 

      Just my opinion. 

      Sent from my iPad

      On Jun 19, 2013, at 1:20 PM, Adrian Howard <adrianh@...> wrote:

       


      On 19 June 2013 21:11, Anders Ramsay <andersr@...> wrote:
      It appears it is necessary to join that list to read the message.  Can you maybe share a bit about what you found serendipitous?

      Bugger. Sorry. Thought it was public to all.

      Basically thread started on "why such a focus on intent driven design? Can designers
      become catalysts of serendipity?". Which led to me mentioning lean ux. Which led to some lean ux push back. Which led to some interesting places on roles 'n' stuff.

      I don't feel happy coping in other folks text without permission - but here's some of the stuff I've been posting:

      ---

      I'm not arguing against basic research - honest! Love basic research to bits. Didn't even mention research ;-) 

      What I don't like is the way research is integrated into the product design and development process as a whole.

      Let me rephrase without using the L-word or "up front" and see if I make any more sense.

      These are things I've seen repeatedly fail:

      1) Building products without doing any research on the customers, the market, the problem, etc. Because it's seen as waste, or too expensive, or folk don't even really know that this kind of work exists. Obviously these usually fail, or at best only partially succeed through luck. This group forms the *vast* majority of product development work I encounter.

      2) People doing sh*t research. Interviews that are sales pitches. God awful surveys. People just making stuff up. This is the second largest group of product development folk I encounter. Similar results to (1).

      3) Finally you get folk doing some quality deep research that produces a ton of fascinating and significant insights. There's a wonderful show reel of interview snippets, a fantastic slide deck and a 200 page report that points to ways that this stuff can be used effectively, along with a set of gorgeous artefacts and prototypes to help communicate the issues that were discovered. Everybody looks very pleased and congratulates themselves on a job well done. Then one or more of the following happens:

      - It takes so long, and is so resource intensive, that it gets done rarely so most project work falls back to (1) or (2) above anyway.

      - It all sits on a shelf and gets ignored, then forgotten, since it's not actionable in current development cycle

      - It gets handed over to a group of product managers, who then brief the design group, who the produce a spec for the dev team, which then gets cut for scope, and then readjusted with the feedback from the last product release, and ends up producing a mess that bears no relation to the original work.

      - it finally gets referenced N months/years after the original research was done, after much of the detail and meat has been forgotten or reinterpreted, and the world has changed enough for the work to be largely invalid (fond memories of working with $LargeCompany that was still trying to apply the ethno work it did on mobile phone usage in 1998 in 2005...)

      - Just after the research is done the iPhone gets launched, or everybody starts using twitter, or 9/11 happens, or $competitor launches something smack bang in the middle of the space you were researching or ... - and suddenly people start doing things differently...

      - Just after the research is done (or even better - half way through ;-) the company decides that it's focus is now $somethingelse.

      - Sometimes even the best research misses something, or makes a mistake, or over/under generalises. When this eventually surfaces with the user testing of $prototype the research folk are never around to help figure out what's happening.

      Because the research isn't an ongoing process (so misses interesting stuff that happens outside the research period), because it's not valued by the organisation as a whole, because it takes too long to get to the point where it can be applied - this great research work is wasted. Which sucks. 

      Especially when it's me wasting my time with the research. 

      What I've seen work better are patterns like this:

      * Research isn't seen as a special one-off activity. Understanding and learning about the customers is seen as a core activity that's done all of the time. It's always running a continuous drip-feed of insights into the organisation.

      * More people do user research. Even if I have to spend a stack of my time training folk and facilitating that research. Because getting more people involved frees me up to spend more time on the hard stuff. Because the 80/20 rule applies even for user research. Because even if those folk make mistakes we're doing research all of the time - so we catch it later on. Because it's much easier for folk to grok that research if they actually see where it came from and are invested in it from the start.

      * Feed the results from our research activities to the rest of the organisation continuously. Refine them continuously. Validate them continuously. Build that connective tissue you talk about continuously. Design the process so that we can deal with uncertain and tentative results by revisiting them regularly. Enable the organisation to act on stuff early enough to be effective.

      Because the focus of the whole team is on learning and discovery, and because everybody is trying to do that continually, it's much, much easier (in my experience anyway) to take advantage of those serendipitous events that happen to pass in front of us. "Chance favors the prepared mind" as Pasteur says.

      (Okay... brace yourself I'm gonna start use the L-word again.... Deep calming breath. Deep calming breath... ;-)

      And this, for me anyway, is what the Lean UX folk are talking about. It's certainly what I'm talking about when I use the phrase. It's doing *more* research over the lifetime of a project - not less. It's about getting *more* people involved with doing research, so it's valued and appreciated and acted upon more effectively.

      I'll happily agree that many folk in the Lean Startup community are doing a bloody awful job of it. They're doing very little observational work, and their approach to their customers is often deeply broken - based on advice from folk with a sales background rather than anthro/ethno/ux folk. But they are - in my experience anyway - really open to ways of doing it better, and quick studies to boot.

      Their whole *process* is built around learning. If folk have bought into Lean Startup then selling them on vaguely decent user research is way, *way* easier than folk with more traditional approaches to product development. I don't have to convince them that doing research is good - I just have to get them to do it in smarter ways.

      I get a buzz from doing research and understand the customer and their needs. I get an even bigger buzz by being able to use that understanding to make the product better and the customers happy. The Lean UX/Startup stuff has been getting me closer to my happy place over the last couple of years ;-)

      ---

      Cheers,

      Adrian
      --
      adrianh@... / +44 (0)7752 419080 / @adrianh / quietstars.com
      Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S

    • Adrian Howard
      ... I d actually quite strongly disagree with that. Research work not directed as specific product work often helps surface pain points and problems in the
      Message 2 of 15 , Jun 19, 2013
      • 0 Attachment
        On 20 June 2013 07:16, Jon Innes <jinnes@...> wrote:
        Research in the absence of design considerations has no value in a product oriented company. If you want to do that type of work go apply for an academic position or a government grant. 

        I'd actually quite strongly disagree with that. Research work not directed as specific product work often helps surface pain points and problems in the market that research aimed at a particular product will miss. 

        One of the things that causes companies to be blind-sided is that they only do research in the context of their products. They then get stomped all over by $competitor who goes and understands the wider context of their customer's problems.

        You need both if you're going to be successful in the long term (IMHO anyway ;)

        Cheers,

        Adrian
        --
        adrianh@... / +44 (0)7752 419080 / @adrianh / quietstars.com
        Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S
      • Jon Innes
        Adrian, I think we actually agree here. Understanding customer s problems is the key to design. That s not what I was talking about. I just question folks who
        Message 3 of 15 , Jun 20, 2013
        • 0 Attachment
          Adrian,

          I think we actually agree here. Understanding customer's problems is the key to design. That's not what I was talking about. I just question folks who do research with no idea how it applies to anything.

          If you don't have a target market or product idea in mind, then you should stay out of business.

          On Jun 19, 2013, at 11:43 PM, Adrian Howard wrote:

           


          On 20 June 2013 07:16, Jon Innes <jinnes@...> wrote:
          Research in the absence of design considerations has no value in a product oriented company. If you want to do that type of work go apply for an academic position or a government grant. 

          I'd actually quite strongly disagree with that. Research work not directed as specific product work often helps surface pain points and problems in the market that research aimed at a particular product will miss. 

          One of the things that causes companies to be blind-sided is that they only do research in the context of their products. They then get stomped all over by $competitor who goes and understands the wider context of their customer's problems.

          You need both if you're going to be successful in the long term (IMHO anyway ;)

          Cheers,

          Adrian
          --
          adrianh@... / +44 (0)7752 419080 / @adrianh / quietstars.com
          Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S


        • Adrian Howard
          ... Can you give some examples of what you thinking about then - since I m not sure what you re arguing against? Something seemed to have struck a nerve ;-) To
          Message 4 of 15 , Jun 20, 2013
          • 0 Attachment
            On 20 June 2013 09:46, Jon Innes <jinnes@...> wrote:
            I think we actually agree here. Understanding customer's problems is the key to design. That's not what I was talking about. I just question folks who do research with no idea how it applies to anything.

            If you don't have a target market or product idea in mind, then you should stay out of business.

            Can you give some examples of what you thinking about then - since I'm not sure what you're arguing against? Something seemed to have struck a nerve ;-)

            To me it feels like you're tilting against a straw man... I don't know of anybody in industry (in academia either come to that) who just able around observing things with no purpose in mind. 

            Cheers,

            Adrian

            --
            adrianh@... / +44 (0)7752 419080 / @adrianh / quietstars.com
            Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S
          • Tim Wright
            On the other hand, lots of successfull products have been created by accident. The person who noticed them might not have had a product idea or market in
            Message 5 of 15 , Jun 20, 2013
            • 0 Attachment

              On the other hand, lots of successfull products have been created by accident. The person who noticed them might not have had a "product idea or market in mind" - but they saw something and realised that there was one.

              My favourite was the person who was trying to cure malaria and accidentally invented a new colour - Mauve.


              Tim


              On 20 June 2013 20:51, Adrian Howard <adrianh@...> wrote:
               


              On 20 June 2013 09:46, Jon Innes <jinnes@...> wrote:
              I think we actually agree here. Understanding customer's problems is the key to design. That's not what I was talking about. I just question folks who do research with no idea how it applies to anything.

              If you don't have a target market or product idea in mind, then you should stay out of business.

              Can you give some examples of what you thinking about then - since I'm not sure what you're arguing against? Something seemed to have struck a nerve ;-)

              To me it feels like you're tilting against a straw man... I don't know of anybody in industry (in academia either come to that) who just able around observing things with no purpose in mind. 

              Cheers,

              Adrian

              --
              Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S


            • Jon Innes
              The tech industry is full of people who get paid to do basically useless work, including research that leads to nothing of value. They aren t accountable for
              Message 6 of 15 , Jun 21, 2013
              • 0 Attachment
                The tech industry is full of people who get paid to do basically useless work, including research that leads to nothing of value. They aren't accountable for anything tangible. They claim to be trying to improve things, but can't measure their impact.

                That said I agree with Tim that lots of successful products are the result of unexpected insights. When I worked for Scott Cook of Intuit his favorite example of this was Malcom McLean's invention of the metal shipping container which was based on his observations as a truck driver. But that's insight of an entrepreneur/designer working in non-research role, which is different than being a paid researcher on staff that produces nothing that ever amounts to value. 

                The "nerve" Adrian struck was related to the idea that doing "research" without a hypothesis or a goal is something one should be paid to do. There are too many "researchers" that give the field a bad name in industry.


                On Jun 20, 2013, at 12:40 PM, Tim Wright wrote:

                 


                On the other hand, lots of successfull products have been created by accident. The person who noticed them might not have had a "product idea or market in mind" - but they saw something and realised that there was one.

                My favourite was the person who was trying to cure malaria and accidentally invented a new colour - Mauve.


                Tim


                On 20 June 2013 20:51, Adrian Howard <adrianh@...> wrote:
                 


                On 20 June 2013 09:46, Jon Innes <jinnes@...> wrote:
                I think we actually agree here. Understanding customer's problems is the key to design. That's not what I was talking about. I just question folks who do research with no idea how it applies to anything.

                If you don't have a target market or product idea in mind, then you should stay out of business.

                Can you give some examples of what you thinking about then - since I'm not sure what you're arguing against? Something seemed to have struck a nerve ;-)

                To me it feels like you're tilting against a straw man... I don't know of anybody in industry (in academia either come to that) who just able around observing things with no purpose in mind. 

                Cheers,

                Adrian

                --
                Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S




              • Natalie Hanson
                Hi all - Since I am the list owner and moderator for anthrodesign, perhaps I can chime in here à especially in regards to Jon s comment The nerve Adrian
                Message 7 of 15 , Jun 21, 2013
                • 0 Attachment
                  Hi all -

                  Since I am the list owner and moderator for anthrodesign, perhaps I can chime in here … especially in regards to Jon's comment "The "nerve" Adrian struck was related to the idea that doing "research" without a hypothesis or a goal is something one should be paid to do. There are too many "researchers" that give the field a bad name in industry."  Yes, there is lots of lame research, just as there is lots of crappy agile … but that's not an accurate representation of what they are frustrated about.  

                  Anthrodesign is a listserv and community focused on the use of ethnographic methods in business.  Those methods have the reputation of being long and expensive (though it doesn't need to be the case!).  Those who participate in the list are extremely frustrated with Lean experiences / practitioners that they've encountered.  In some cases they are conflating Lean and Agile, but the theme is basically the same … which is … how can they execute meaningful, valuable research an an environment which is focused on cost-cutting?  My answer is that if Lean is being practiced properly that it's way more nuanced than that.  I referenced a blog post I wrote, which is a sneak peek at a book chapter on the same topic - http://www.nataliehanson.com/2013/02/21/lean-ux-again/.  Those who have spoken up feel that US businesses are committed to the cost-cutting, efficiency part of Lean and not more.  Adrian and I have been trying to help them understand that Lean has some wonderful synergies with UX, and that it's not all bad.  Unfortunately, that hasn't been their experience.  

                  Regarding the research … in an agile environment I'm learning that it is extremely challenging to deliver insightful, usable research.  So rather than just criticizing, how about working together to make all members of the team effective contributors?  I currently manage a centralized UX team in a software development organization; the engineers with front-end development as their primary focus also report to me.  Right now in my team we're experimenting with some blending across roles (design/dev for HTML and CSS) but also research and design.  I am trying to put our researchers out in front at R0 to look a the work practices of specific user types so that we can effectively write their user stories.  And the designers are doing more of the incremental interviewing and validation so we stay connected to the release and sprint activities.  It is not at all obvious how to do it well … but I am fortunate to work in a team where there is a lot of mutual respect and discussion and continuous adjustment so that we can make sure that all the people on the team are adding value with their unique abilities, and at the same time making clear the UX contributions to what ships.  But that's only possible because we're working towards those improvements across all organizational functions.

                  Natalie


                  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * 
                  Natalie Hanson, PhD
                  ndhanthro at yahoo dot com

                  On Jun 21, 2013, at 2:29 am, Jon Innes <jinnes@...> wrote:

                   

                  The tech industry is full of people who get paid to do basically useless work, including research that leads to nothing of value. They aren't accountable for anything tangible. They claim to be trying to improve things, but can't measure their impact.

                  That said I agree with Tim that lots of successful products are the result of unexpected insights. When I worked for Scott Cook of Intuit his favorite example of this was Malcom McLean's invention of the metal shipping container which was based on his observations as a truck driver. But that's insight of an entrepreneur/designer working in non-research role, which is different than being a paid researcher on staff that produces nothing that ever amounts to value. 

                  The "nerve" Adrian struck was related to the idea that doing "research" without a hypothesis or a goal is something one should be paid to do. There are too many "researchers" that give the field a bad name in industry.


                  On Jun 20, 2013, at 12:40 PM, Tim Wright wrote:

                   


                  On the other hand, lots of successfull products have been created by accident. The person who noticed them might not have had a "product idea or market in mind" - but they saw something and realised that there was one.

                  My favourite was the person who was trying to cure malaria and accidentally invented a new colour - Mauve.


                  Tim


                  On 20 June 2013 20:51, Adrian Howard <adrianh@...> wrote:
                   


                  On 20 June 2013 09:46, Jon Innes <jinnes@...> wrote:
                  I think we actually agree here. Understanding customer's problems is the key to design. That's not what I was talking about. I just question folks who do research with no idea how it applies to anything.

                  If you don't have a target market or product idea in mind, then you should stay out of business.

                  Can you give some examples of what you thinking about then - since I'm not sure what you're arguing against? Something seemed to have struck a nerve ;-)

                  To me it feels like you're tilting against a straw man... I don't know of anybody in industry (in academia either come to that) who just able around observing things with no purpose in mind. 

                  Cheers,

                  Adrian

                  --
                  Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S






                • davidkallen
                  Natalie, I agree with everything you wrote in that post, with one adjustment: Your company are fortunate to have YOU lead the team, creating a climate where
                  Message 8 of 15 , Jun 22, 2013
                  • 0 Attachment
                    Natalie, I agree with everything you wrote in that post, with one adjustment: Your company are fortunate to have YOU lead the team, creating a climate where mutual respect exists, and cross-functional innovation towards a common goal is invited. That is the climate needed to effectively blend the long-term research with the tactical agility and deliver high-quality products quickly. And this climate does not occur by accident. It is created by leaders like you, supported by team members who believe in that common vision. And it can only continue if your management has the vision to support you.

                    Our team struggles with the same challenges of blending the different worlds of the UX/IA designer with the agile software developer. Even the UX/IA world has its own blend of hypothesis-driven science and intuition-driven art and aesthetics. Is it any wonder this is hard for teams to achieve? We must blend art, science, business, and engineering into a unified product. To do this, we must blend together teams whose passion ranges across all those areas. It's a miracle that we pull together and achieve this in even small part.


                    --- In agile-usability@yahoogroups.com, Natalie Hanson <ndhanthro@...> wrote:
                    > Those methods have the reputation of being long and expensive (though it doesn't need to be the case!).
                    ...
                    > I currently manage a centralized UX team in a software development organization;
                    > … but I am fortunate to work in a team where there is a lot of mutual respect and ...
                  • Adrian Howard
                    ... I m going to push back at that a little bit. How much of that problem is due to the research or the researcher being useless. How much of it is down to the
                    Message 9 of 15 , Jun 22, 2013
                    • 0 Attachment
                      On 21 June 2013 08:29, Jon Innes <jinnes@...> wrote:
                      The "nerve" Adrian struck was related to the idea that doing "research" without a hypothesis or a goal is something one should be paid to do. There are too many "researchers" that give the field a bad name in industry.

                      I'm going to push back at that a little bit.

                      How much of that problem is due to the research or the researcher being useless. How much of it is down to the organisation being structured in a way that cannot effectively apply the research.

                      As I said before - research without *any* goal or hypothesis feels kind of straw mannish to me. Can't recall I've ever seen it. What *exactly* are we talking about here?

                      Folk who are just researching "the customers"? 

                      In which case, when done well with the right environment, that more general ethnographic work is stupidly useful. It can be where the new product ideas and hypothesis come from.

                      Where it's completely useless and wasteful is when it's done in a context where it's not going to be used. If the work that I do just ends up as a shiny 100 page report on the CEOs bookshelf then it's a waste of time and space - however intellectually satisfying.

                      Sometimes that report sitting on the shelf is the researcher's fault. Sometimes it's the organisation's fault. Most of the time I would imagine that it's a bit of both.

                      So are the problems you're seeing with "research", "researchers" or "organisations" (or all three)?

                      Cheers,

                      Adrian
                      -- 
                      adrianh@... / +44 (0)7752 419080 / @adrianh / quietstars.com
                      Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S

                    • Jared Spool
                      ... Researchers are not immune to Sturgeon s Law. 90% of all research projects, in my experience, are crap. They are poorly formulated, poorly executed, and
                      Message 10 of 15 , Jun 23, 2013
                      • 0 Attachment

                        On Jun 22, 2013, at 3:47 PM, Adrian Howard wrote:

                        As I said before - research without *any* goal or hypothesis feels kind of straw mannish to me. Can't recall I've ever seen it. What *exactly* are we talking about here?

                        Folk who are just researching "the customers"? 


                        Researchers are not immune to Sturgeon's Law. 90% of all research projects, in my experience, are crap. They are poorly formulated, poorly executed, and poorly integrated with the rest of the organization.

                        Well done research (the remaining 10%) starts with a focus. Sometimes the focus is narrow ("We need to see if this feature is implemented the best it can be."); sometimes it's broad ("I wonder what we could do to help our customers better."). Whether narrow or broad (or something in between), the focus guides the research.


                        Where it's completely useless and wasteful is when it's done in a context where it's not going to be used. If the work that I do just ends up as a shiny 100 page report on the CEOs bookshelf then it's a waste of time and space - however intellectually satisfying.

                        Sometimes that report sitting on the shelf is the researcher's fault. Sometimes it's the organisation's fault. Most of the time I would imagine that it's a bit of both.

                        So are the problems you're seeing with "research", "researchers" or "organisations" (or all three)?

                        Much of the poorly executed research I see happens because the organization's reward system and culture have not been adjusted to accept it. If an organization isn't set up to take the research and its results in (which are separate things), then you get the result of the shiny report on the CEO desk. (Ironically, the best research never has a report to put on the CEOs desk, which is fine, because the CEO was involved in the work throughout.)

                        You can't separate the problems with being with "research", "researchers", or "organisations", in my opinion. They are deeply integrated.

                        Jared



                      • John Schrag
                        ... When I teach the basics of doing research (usually to people who have already decided to put out a survey) I recommend starting with two questions: What
                        Message 11 of 15 , Jun 23, 2013
                        • 0 Attachment
                          On 2013-06-23, at 9:30 AM, Jared Spool wrote:
                          Researchers are not immune to Sturgeon's Law. 90% of all research projects, in my experience, are crap. They are poorly formulated, poorly executed, and poorly integrated with the rest of the organization..


                          When I teach the basics of doing research (usually to people who have already decided to put out a survey)  I recommend starting with two questions:  What decisions do you need to make?  What information do you need to make those decisions?  Only after those two questions are settled, do I help them figure out how to collect that information.  I'm frequently amazed at how much resistance there is to setting a clear research question, even by the people who are asking for the research.  (And almost always a survey is not the right way, but they're "easy to do". . .)

                          John
                        • Adrian Howard
                          Hey Jared, On 23 June 2013 14:30, Jared Spool wrote: [snip] ... This. Times a bazillion. ... Indeed - in fact I ll just copy n paste something
                          Message 12 of 15 , Jun 23, 2013
                          • 0 Attachment
                            Hey Jared,

                            On 23 June 2013 14:30, Jared Spool <jspool@...> wrote:
                            [snip]
                            Much of the poorly executed research I see happens because the organization's reward system and culture have not been adjusted to accept it. If an organization isn't set up to take the research and its results in (which are separate things), then you get the result of the shiny report on the CEO desk. (Ironically, the best research never has a report to put on the CEOs desk, which is fine, because the CEO was involved in the work throughout.)

                            This. Times a bazillion.
                             
                            You can't separate the problems with being with "research", "researchers", or "organisations", in my opinion. They are deeply integrated.

                            Indeed - in fact I'll just copy'n'paste something I wrote yesterday on the anthrodesign list

                            ---- [snip start] ----

                            I no longer judge my success on the quality of my findings. 

                            I judge it on the effect of my findings.

                            It doesn't matter how glorious the insights. If they don't end up making the organisation, the product, the world better they're a failure. If management, developers, designers, whoever ignores 'em - that's my fault. Not theirs. I've not laid the groundwork for those results to be used effectively. I've delivered something they cannot use or apply. I've solved a different problem from the one that needed solving. 

                            I'm not saying that if an organisation cannot accept or work with "deep" insights I shouldn't do the research. I'm saying that in that situation the first order of the day is to help the organisation move to a place where it can get value. 

                            Until I do - doing the research is waste.

                            ---- [snip end] ----

                            Cheers,

                            Adrian
                            -- 
                            adrianh@... / +44 (0)7752 419080 / @adrianh / quietstars.com
                            Subscribe to the latest Agile & Lean UX news here > http://is.gd/KREt5S

                          Your message has been successfully submitted and would be delivered to recipients shortly.