Loading ...
Sorry, an error occurred while loading the content.

1173Re: users don't self-report reliably

Expand Messages
  • aacockburn
    May 5, 2005
    • 0 Attachment
      It just dawned on me why I wouldn't have noticed or cared about the
      first example you gave, Hugh --- I show up to write use cases, and we
      need to write a use case for any system function whether it is used
      hourly or once a blue moon. This is a crucial difference
      in "requirements gathering" (what exactly should the system do)
      and "envisioning" and "usability" (how and how much does it improve
      the life of the user/organization).

      In envisioning and usability, frequency and the related ROI matter
      hugely. In writing a functional spec, we just need to know that it
      gets done ever. Thus, if users tell me this use case is the 60% item,
      I'll make a note of that for other people on the project to worry
      about, and collect the use case. If they tell me this use case is
      just to handle the odd-ball situation, I'll make a note of that for
      other people on the project to worry about, and collect the use
      case. My goal is to capture /all/ and not miss one. Your goal is to
      identify the ROI factor.


      The second case below is different. That falls into
      my "capture /all/" bucket, and so I am all ears listening and
      questioning, looking for all the infrequent, oddball, not-officially-
      supported pathways. That's where I get a clam-level tipoff... and at
      that point I head straight for the surface and above, asking them
      what is on their minds, why they would be doing that --- I don't try
      to deduce what is on their minds from the clam-level data - I simply
      ask them. And my experience here (without having any other reference
      point) is that they DO report reliably in this situation. ...

      Consider, if you will the second case you report on: "we saw people
      doing something called "roll transfers"-tossing a stack of wafers
      from one carrier to another without using the machine that was
      especially designed for the purpose-in violation of procedure. We
      might, with good facilitation have discovered that people break the
      rules this way. We would not, I think, have understood why people
      risk their jobs (or at least their performance appraisals) to do this
      kind of transfer when they don't have to. "

      So I'll offer that the reverse is true of what you just wrote: that
      using facilitatin it is hard to DETECT that situation - role playing
      and observation are helpful here - but having detected it, we can
      UNDERSTAND the 'why' using facilitation.

      Here's my matching example... we're talking about giving medications
      in a hospital using a new barcode reader gadget. We're showing some
      draft use cases to a team that had done a beta test with some hand-
      held / bar code reader device for a month. We're talking about
      scanning the meds at the bed side, and one of the nurses says, "Do we
      have to be at the bedside? I used to carry a bar code of patient in
      my pocket and scan everything in the prep room..." We all got stuck
      at that moment, because it was clear that she was circumventing
      something fundamental in the way that system had been set up (to be
      done only at the patient bedside). After some hemming and hawing in
      the room, we asked her, "Why would you be doing that? If we're going
      to design a new system, it should support what you're trying to do,
      so what were you trying to do?" And she said, "It's to save face in
      front of the patient - I don't want to get to the bed and find I have
      the wrong dose or something." After which, the other nurses agreed to
      this.

      What was serendipidous was her mentioning it. What wasn't
      serendipidous was her being able to say why she was doing it. I don't
      think we needed weeks of observation and note taking to deduce this;
      it was enough just to ask her.

      We have since written a use case called "Optionally pre-check the
      meds", and are finding it popular among the nursing staff in the
      various hospitals once they read and are reminded what it is for
      (that there is no actual med administration going on, it is only
      for "peace of mind"). And it has triggered in their minds various
      situations that matter to us about where they might be standing with
      this that or the other medication needing this that or the other bit
      of procedure, so that we can write the requirements to support ALL
      the places and behaviors they will need (however rare), while
      prohibiting the specific ones the law and the hospital prohibits.

      Reminder: we were lucky in this, because we got to talk to a team
      that had used a scanner for a month, so we had their memories to work
      with. I don't think we would have discovered this 'optionally
      recheck' use case without those memories. The people who originally
      designed that scanner system didn't have that luxury.

      Thus I motivate the need for clam-level details, and that these are
      hard to come by in facilitated sessions (ergo, the facilitator has to
      deliberately work out ways to get clam-level stuff to pop out).

      However, I argue that facilitation works fine for discovering the
      Why, and that self-reporting is quite reliable here.

      Alistair




      --- In agile-usability@yahoogroups.com, "Hugh Beyer" <beyer@i...>
      wrote:
      > On practically the first project we worked on as a business, the
      user
      > population consisted of system managers, who uniformly reported
      that 60-70%
      > of their work was focused on diagnosis and troubleshooting of
      failing
      > systems. CI field interviews revealed that maybe 10-20% of their
      work was
      > diagnosis and troubleshooting. Most of their time was spent on
      routine
      > maintenance tasks. We've found this is typical-users can't tell you
      how they
      > allot their time really-they put too much emphasis on what is
      the "real"
      > job, the interesting, valuable, knowledge work, and overlook the
      time spent
      > on "unimportant" parts of the job.
      >
      >
      >
      > On another project, we were studying chip fabrication. We were told
      how
      > rigid all their requirements were for handling the wafers and how
      carefully
      > they were followed. But when we observed, we saw people doing
      something
      > called "roll transfers"-tossing a stack of wafers from one carrier
      to
      > another without using the machine that was especially designed for
      the
      > purpose-in violation of procedure. We might, with good facilitation
      have
      > discovered that people break the rules this way. We would not, I
      think, have
      > understood why people risk their jobs (or at least their performance
      > appraisals) to do this kind of transfer when they don't have to.
      >
      >
    • Show all 23 messages in this topic