Loading ...
Sorry, an error occurred while loading the content.
 

RE: [agile-usability] Digest Number 299

Expand Messages
  • Gardner, Jaynee B
    Wow! What great discussions! Let me add one illustration to make a point: I was hired by a manufacturer to fix a problem with numerous data entry clerks who
    Message 1 of 5 , Dec 8, 2005
      Wow! What great discussions! Let me add one illustration to make a
      point: I was hired by a manufacturer to fix a problem with numerous data
      entry clerks who were getting RSIs at a very high rate. The company had
      an automated system that tracked materials received into the plant, and
      another automated scheduling system that management used to monitor the
      flow of deliverable product. The data entry clerks printed out lists of
      items in the materials tracking system, lists of orders from an external
      system, and lists of internal items generated from the business office.
      Then, using these lists, they filled out an electronic form in the
      automated scheduling system, so that the manufacturing floor would
      receive a report showing them where to position the materials for
      building product, and management could track productivity. A team of 18
      clerks was experiencing sore necks, sore backs, carpal tunnel syndrome,
      eye strain, sore shoulders, and lots of stress because they were always
      behind and they were being blamed for holding up production rates.
      Proposed solutions: better keyboards (split keyboard, vertical
      keyboard, tilt keyboards, adjustable keyboard trays, etc.); better mice
      (track-balls, vertical mouse, touch pads, etc.); better layout of the
      data entry screen (better instructions, order the data entry to match
      the order of the printed material, larger fields, tab-less jump from
      field to field, etc.); better ergonomics at the work-station (adjust the
      chair, alter monitor height, position the work material, task lighting,
      etc.); alter the input material (larger font size, colored lines,
      friendlier visual page layout, etc.). The entire standard Usability
      bag-o-tricks had already been thrown at this problem, and still the
      clerks were dropping like flies. Some additional "out there" solutions
      had also been mentioned (including voice inputs) and discarded.

      The creative solution was not what I would consider "whiz-bang" or on
      the cutting edge of technology, since we have been doing it since the
      1950's at least. I simply proposed to the company that they require
      their systems to re-use every element of data that was already known to
      any system, and dismantle the beleaguered data entry team altogether.

      The cost was enormous! I was proposing that they customize existing
      software. I was requiring integration between software from different
      companies. There was no open architecture on this software. There was no
      source code. There was no way this was going to happen. One year and
      millions of dollars in disability claims, workstation improvements,
      process improvements (anybody priced a Kaizen lately?), and productivity
      loss later, the company finally decided to try my solution. And it
      worked.

      The point is, there is no one-size-fits-all solution in the HFE
      bag-o-tricks; and you cannot make assumptions about a solution based on
      its apparent cost alone. Please, please don't fall into the trap of
      thinking that "Creative" equals "Costly" or "Whiz-bang". As Larry
      already pointed out, if we are to call ourselves engineers, then just
      the opposite must be true: "creative" must fit into program constraints.
      Task analysis is essential. Usability testing is essential. (sorry about
      the ambiguous use of the word "pitch" in my previous post - I realized
      upon re-reading that a person could interpret "pitch" as "throw away",
      whereas my intended use of "pitch" was "propose/promote").

      How does that fit with Agility? I am still trying to figure that out. I
      am new to the Agile portion of this. That's why I'm here. So far, not
      disappointed in the discussion list at all and truly enjoying all of the
      discourse.

      Jaynee Gardner, PhD, CHFP
      PMA Product Manager
      JSF, ALIS SEIT
      (817) 935-3562
    • Jamie Nettles
      What a great story. Part of the Manifesto for Agile Software Development states Simplicity-the art of maximizing the amount of work not done-is essential.
      Message 2 of 5 , Dec 8, 2005
        What a great story. Part of the Manifesto for Agile Software
        Development states "Simplicity-the art of maximizing the amount of work
        not done-is essential."

        Jamie Nettles

        -----Original Message-----
        From: agile-usability@yahoogroups.com
        [mailto:agile-usability@yahoogroups.com] On Behalf Of Gardner, Jaynee B
        Sent: Thursday, December 08, 2005 1:18 PM
        To: agile-usability@yahoogroups.com
        Subject: RE: [agile-usability] Digest Number 299

        Wow! What great discussions! Let me add one illustration to make a
        point: I was hired by a manufacturer to fix a problem with numerous data
        entry clerks who were getting RSIs at a very high rate. The company had
        an automated system that tracked materials received into the plant, and
        another automated scheduling system that management used to monitor the
        flow of deliverable product. The data entry clerks printed out lists of
        items in the materials tracking system, lists of orders from an external
        system, and lists of internal items generated from the business office.
        Then, using these lists, they filled out an electronic form in the
        automated scheduling system, so that the manufacturing floor would
        receive a report showing them where to position the materials for
        building product, and management could track productivity. A team of 18
        clerks was experiencing sore necks, sore backs, carpal tunnel syndrome,
        eye strain, sore shoulders, and lots of stress because they were always
        behind and they were being blamed for holding up production rates.
        Proposed solutions: better keyboards (split keyboard, vertical
        keyboard, tilt keyboards, adjustable keyboard trays, etc.); better mice
        (track-balls, vertical mouse, touch pads, etc.); better layout of the
        data entry screen (better instructions, order the data entry to match
        the order of the printed material, larger fields, tab-less jump from
        field to field, etc.); better ergonomics at the work-station (adjust the
        chair, alter monitor height, position the work material, task lighting,
        etc.); alter the input material (larger font size, colored lines,
        friendlier visual page layout, etc.). The entire standard Usability
        bag-o-tricks had already been thrown at this problem, and still the
        clerks were dropping like flies. Some additional "out there" solutions
        had also been mentioned (including voice inputs) and discarded.

        The creative solution was not what I would consider "whiz-bang" or on
        the cutting edge of technology, since we have been doing it since the
        1950's at least. I simply proposed to the company that they require
        their systems to re-use every element of data that was already known to
        any system, and dismantle the beleaguered data entry team altogether.

        The cost was enormous! I was proposing that they customize existing
        software. I was requiring integration between software from different
        companies. There was no open architecture on this software. There was no
        source code. There was no way this was going to happen. One year and
        millions of dollars in disability claims, workstation improvements,
        process improvements (anybody priced a Kaizen lately?), and productivity
        loss later, the company finally decided to try my solution. And it
        worked.

        The point is, there is no one-size-fits-all solution in the HFE
        bag-o-tricks; and you cannot make assumptions about a solution based on
        its apparent cost alone. Please, please don't fall into the trap of
        thinking that "Creative" equals "Costly" or "Whiz-bang". As Larry
        already pointed out, if we are to call ourselves engineers, then just
        the opposite must be true: "creative" must fit into program constraints.
        Task analysis is essential. Usability testing is essential. (sorry about
        the ambiguous use of the word "pitch" in my previous post - I realized
        upon re-reading that a person could interpret "pitch" as "throw away",
        whereas my intended use of "pitch" was "propose/promote").

        How does that fit with Agility? I am still trying to figure that out. I
        am new to the Agile portion of this. That's why I'm here. So far, not
        disappointed in the discussion list at all and truly enjoying all of the
        discourse.

        Jaynee Gardner, PhD, CHFP
        PMA Product Manager
        JSF, ALIS SEIT
        (817) 935-3562





        ------------------------ Yahoo! Groups Sponsor --------------------~-->
        Most low income households are not online. Help bridge the digital
        divide today!
        http://us.click.yahoo.com/I258zB/QnQLAA/TtwFAA/dpFolB/TM
        --------------------------------------------------------------------~->


        Yahoo! Groups Links
      • Gardner, Jaynee B
        Hey Discussion List Folks, here is a scheme for Usability in our new Agile process that I am developing. All comments are welcome. We do 4-week sprints (I
        Message 3 of 5 , Dec 8, 2005
          Hey Discussion List Folks, here is a scheme for Usability in our new
          Agile process that I am developing. All comments are welcome.

          We do 4-week sprints (I don't control that, it is what it is). We have
          negotiated G/O/SC for our releases and sprints for the individual system
          products (my product, a portable electronic device intended to aid
          aircraft maintainers, is a super-system product, that is, it integrates
          numerous other system products into a hardware device). I am still
          negotiating G/O/SC for my product. Yes, that's backward, since my G/O/SC
          ought to influence the G/O/SC of the systems on my product (I don't
          control that, either). My product G/O/SC is very Usability-oriented,
          since I am by practice and education an HFE. The rest of our product
          owners will most likely "borrow" my usability-oriented G/O/SC because
          it's easier than making up new stuff.

          At sprint planning, I (product owner) present a User story regarding how
          my product will be used. It's actually part of a Usage Profile in the
          Concept of Operations (Conops) document, and there are standard elements
          of this Usage profile that get transferred or assumed in the sprint User
          story. The story states what the user is trying to accomplish (for
          example: replenish all consumables in Aircraft A), some performance
          parameters (get it done in X amount of time to support Aircraft
          turn-around), some quality constraints (all personnel and data must
          remain uncompromised), and some additional criteria for ensuring that
          success has been attained (system reports that Aircraft configuration is
          correct and in the flyable condition). The Usage Profile includes such
          additional material as maintainer goals (do it quickly, do it right, do
          it safely), maintainer needs (work instruction, task instruction,
          technical data), and maintainer activities (walking, reaching, bending,
          stooping, eyes focused 'here', hands engaged 'here', etc.).

          I can present the User story via any means: (in previous programs, not
          Agile) I have had people actually act out the motions for me. I have
          drawn stick figures on a white board. I have made power-point
          prototypes. I have made story-board prototypes. The objective is to
          convey to the team what you want to see happen when they have finished
          this sprint. The sprint team has this User story in front of them
          throughout the sprint, so that they see what the product needs to be at
          the "finish line". The Product Owner should remain in close
          communication with the sprint team throughout the sprint, to reinforce
          the user story, and clarify or offer guidance at any decision points, as
          needed.

          The test team participates in sprint planning to determine the type and
          timing of tests during the sprint. When a unit of code is
          function-tested at the unit-test level, it is simultaneously tested for
          operability at the unit-test level. That is, in addition to the item
          meeting functional requirements, a user must be able to operate the
          basic interface: reach the key, read the label, etc. at the level that
          supports the delivery of that particular function. The validity of the
          function or its delivery is not assessed. The fundamental questions
          being answered at the unit-level are: does this element of the system do
          what we said it would do? And, could a person engage with this system to
          cause it to do what we said it would do?

          When more than one element (or unit) of the sprint product is integrated
          or glued together, an integrated test takes place. The test team
          determines the optimal point in the sprint for this (by schedule or
          other agreed-upon criteria), and it is negotiated and estimated as part
          of sprint planning. Whenever integrated test of a group of functions is
          conducted, then usability of that function group is also assessed, in
          terms of whether or not the user can successfully access that group of
          functions (screen layout, screen flows, automated items vs.
          human-in-the-loop items, entry point into the task flow, time needed to
          read and select from a screen, error rate, error-recovery rate, and just
          the overall "huh?" factor). The success criteria of the test (e.g.,
          "must read and select correct button without making an error 90% of the
          time") is negotiated with inputs from an HFE on the test team.

          Ad hoc Usability tests can take place at any time during a sprint as
          needed. For example, the wording of a specific message to the user could
          be tested for clarity and understanding. Or, rough prototypes of two
          possible implementations could be examined side-by-side to determine
          which one is easier or more effective for the user.

          At the end of the sprint, the deliverable product is demonstrated, and
          its success is assessed against the negotiated success criteria. The
          User story is the script for the demonstration. Specific quality
          criteria, similar to the integrated test criteria but applied at the
          product level, are measured. For example, a Functional sprint success
          criterion might be "system product displays servicing levels of all
          consumables on the aircraft". A Usability sprint success criterion might
          be "User can access knowledge of the servicing level of consumable X,
          with fewer than 2 key presses, with time-on-task less than X seconds,
          without making a screen selection error X% of the time".

          The overall idea is that functional test of any kind, at any level of
          granularity, during or at the end of the sprint, is matched
          pace-for-pace with an associated usability test at the same depth and
          granularity. The intent is that functional test shows that we satisfied
          the requirement, while usability tests shows that we can make that
          functionality available to the user. The concept is that the Agile
          process allows us to place both elements of product delivery
          side-by-side, and that product delivery is not complete until both
          elements (functionality and usability) are satisfied.

          (end of text for comment)

          That's all I've got so far. We are preparing for a go-live date of 1
          January 2006 to switch to Agile processes, and still nobody on our team
          understands how the Usability piece fits in (there is a lot of implicit
          Usability built into the Agile process, but there does not seem to be a
          whole lot of explicit direction). So, in addition to being a Product
          Owner, I've been asked to don my HFE hat and articulate just how we
          think we are going to get better Usability out of our new process (and
          how much will it cost?).

          Your comments are welcome. I realize I have not yet addressed the
          top-level integration of the system products onto my super-system
          product, nor have I addressed going back and assessing the validity of a
          requirement with respect to its ultimate solution. All comments on those
          two topics are welcome as well.

          Thanks in advance for your thoughts and ideas!

          Jaynee Gardner, PhD, CHFP
          PMA Product Manager
        • Frank Maurer
          Jaynee: What you explain in your posting is consistent with what we ve seen other Interaction Designers do in agile teams (see Paul McInerney and my article in
          Message 4 of 5 , Dec 8, 2005
            Jaynee:

            What you explain in your posting is consistent with what we've seen other
            Interaction Designers do in agile teams (see Paul McInerney and my article
            in ACM Interaction in Nov/Dec 2005). They work on the customer side and are
            typically an iteration or so ahead of the development team in coming up with
            a big picture. They then feed this picture into the team in form of user
            stories. They also act as advisors to the team regarding everyday usability
            issues.

            After reading your post, I would have one concern. It sounds like you give
            the team a set of requirements (in form of a user story) and then they are
            supposed to deliver at the end of the sprint. When I observe agile teams,
            the scope of a sprint is not given to them but is negotiated based on the
            customer wishes and effort estimates by the team. Without that element,
            scope is fixed (as well as schedule and cost). As a result quality will
            fluctuate (depending on how good your estimates were regarding development
            effort).

            Frank


            ________________________________

            From: agile-usability@yahoogroups.com
            [mailto:agile-usability@yahoogroups.com] On Behalf Of Gardner, Jaynee B
            Sent: Thursday, December 08, 2005 2:55 PM
            To: agile-usability@yahoogroups.com
            Subject: RE: [agile-usability] Digest Number 299


            Hey Discussion List Folks, here is a scheme for Usability in our new
            Agile process that I am developing. All comments are welcome.

            We do 4-week sprints (I don't control that, it is what it is). We
            have
            negotiated G/O/SC for our releases and sprints for the individual
            system
            products (my product, a portable electronic device intended to aid
            aircraft maintainers, is a super-system product, that is, it
            integrates
            numerous other system products into a hardware device). I am still
            negotiating G/O/SC for my product. Yes, that's backward, since my
            G/O/SC
            ought to influence the G/O/SC of the systems on my product (I don't
            control that, either). My product G/O/SC is very Usability-oriented,
            since I am by practice and education an HFE. The rest of our product
            owners will most likely "borrow" my usability-oriented G/O/SC
            because
            it's easier than making up new stuff.

            At sprint planning, I (product owner) present a User story regarding
            how
            my product will be used. It's actually part of a Usage Profile in
            the
            Concept of Operations (Conops) document, and there are standard
            elements
            of this Usage profile that get transferred or assumed in the sprint
            User
            story. The story states what the user is trying to accomplish (for
            example: replenish all consumables in Aircraft A), some performance
            parameters (get it done in X amount of time to support Aircraft
            turn-around), some quality constraints (all personnel and data must
            remain uncompromised), and some additional criteria for ensuring
            that
            success has been attained (system reports that Aircraft
            configuration is
            correct and in the flyable condition). The Usage Profile includes
            such
            additional material as maintainer goals (do it quickly, do it right,
            do
            it safely), maintainer needs (work instruction, task instruction,
            technical data), and maintainer activities (walking, reaching,
            bending,
            stooping, eyes focused 'here', hands engaged 'here', etc.).

            I can present the User story via any means: (in previous programs,
            not
            Agile) I have had people actually act out the motions for me. I have
            drawn stick figures on a white board. I have made power-point
            prototypes. I have made story-board prototypes. The objective is to
            convey to the team what you want to see happen when they have
            finished
            this sprint. The sprint team has this User story in front of them
            throughout the sprint, so that they see what the product needs to be
            at
            the "finish line". The Product Owner should remain in close
            communication with the sprint team throughout the sprint, to
            reinforce
            the user story, and clarify or offer guidance at any decision
            points, as
            needed.

            The test team participates in sprint planning to determine the type
            and
            timing of tests during the sprint. When a unit of code is
            function-tested at the unit-test level, it is simultaneously tested
            for
            operability at the unit-test level. That is, in addition to the item
            meeting functional requirements, a user must be able to operate the
            basic interface: reach the key, read the label, etc. at the level
            that
            supports the delivery of that particular function. The validity of
            the
            function or its delivery is not assessed. The fundamental questions
            being answered at the unit-level are: does this element of the
            system do
            what we said it would do? And, could a person engage with this
            system to
            cause it to do what we said it would do?

            When more than one element (or unit) of the sprint product is
            integrated
            or glued together, an integrated test takes place. The test team
            determines the optimal point in the sprint for this (by schedule or
            other agreed-upon criteria), and it is negotiated and estimated as
            part
            of sprint planning. Whenever integrated test of a group of functions
            is
            conducted, then usability of that function group is also assessed,
            in
            terms of whether or not the user can successfully access that group
            of
            functions (screen layout, screen flows, automated items vs.
            human-in-the-loop items, entry point into the task flow, time needed
            to
            read and select from a screen, error rate, error-recovery rate, and
            just
            the overall "huh?" factor). The success criteria of the test (e.g.,
            "must read and select correct button without making an error 90% of
            the
            time") is negotiated with inputs from an HFE on the test team.

            Ad hoc Usability tests can take place at any time during a sprint as
            needed. For example, the wording of a specific message to the user
            could
            be tested for clarity and understanding. Or, rough prototypes of two
            possible implementations could be examined side-by-side to determine
            which one is easier or more effective for the user.

            At the end of the sprint, the deliverable product is demonstrated,
            and
            its success is assessed against the negotiated success criteria. The
            User story is the script for the demonstration. Specific quality
            criteria, similar to the integrated test criteria but applied at the
            product level, are measured. For example, a Functional sprint
            success
            criterion might be "system product displays servicing levels of all
            consumables on the aircraft". A Usability sprint success criterion
            might
            be "User can access knowledge of the servicing level of consumable
            X,
            with fewer than 2 key presses, with time-on-task less than X
            seconds,
            without making a screen selection error X% of the time".

            The overall idea is that functional test of any kind, at any level
            of
            granularity, during or at the end of the sprint, is matched
            pace-for-pace with an associated usability test at the same depth
            and
            granularity. The intent is that functional test shows that we
            satisfied
            the requirement, while usability tests shows that we can make that
            functionality available to the user. The concept is that the Agile
            process allows us to place both elements of product delivery
            side-by-side, and that product delivery is not complete until both
            elements (functionality and usability) are satisfied.

            (end of text for comment)

            That's all I've got so far. We are preparing for a go-live date of 1
            January 2006 to switch to Agile processes, and still nobody on our
            team
            understands how the Usability piece fits in (there is a lot of
            implicit
            Usability built into the Agile process, but there does not seem to
            be a
            whole lot of explicit direction). So, in addition to being a Product
            Owner, I've been asked to don my HFE hat and articulate just how we
            think we are going to get better Usability out of our new process
            (and
            how much will it cost?).

            Your comments are welcome. I realize I have not yet addressed the
            top-level integration of the system products onto my super-system
            product, nor have I addressed going back and assessing the validity
            of a
            requirement with respect to its ultimate solution. All comments on
            those
            two topics are welcome as well.

            Thanks in advance for your thoughts and ideas!

            Jaynee Gardner, PhD, CHFP
            PMA Product Manager



            ________________________________

            YAHOO! GROUPS LINKS


            * Visit your group "agile-usability
            <http://groups.yahoo.com/group/agile-usability> " on the web.

            * To unsubscribe from this group, send an email to:
            agile-usability-unsubscribe@yahoogroups.com
            <mailto:agile-usability-unsubscribe@yahoogroups.com?subject=Unsubscribe>

            * Your use of Yahoo! Groups is subject to the Yahoo! Terms of
            Service <http://docs.yahoo.com/info/terms/> .


            ________________________________
          • Desilets, Alain
            The creative solution was not what I would consider whiz-bang or on the cutting edge of technology, since we have been doing it since the 1950 s at least. I
            Message 5 of 5 , Dec 9, 2005
              The creative solution was not what I would consider "whiz-bang" or on
              the cutting edge of technology, since we have been doing it since the
              1950's at least. I simply proposed to the company that they require
              their systems to re-use every element of data that was already known to
              any system, and dismantle the beleaguered data entry team altogether.

              -- Alain:
              Great story!

              What it illustrates is the importance of choosing the right problem to
              solve (in your example, "minimize data re-entry by integrating different
              systems" rather than "make data entry easier by reimplementing the
              interface for existing systems"). This is an area where upfront work
              pays off tremendously. If you choose the wrong problem to solve, you
              will end up with something useless no matter what development
              methodology you use.

              That said, in my experience choosing the right problem to solve is
              usually not that cut and dry (especially when developping highly
              innovative products like in startup companies or in research
              organisations where I work). This is where Agile methodologies can help.

              In your example, suppose cost of integrating different systems was
              estimated to be 100 times higher than the cost of reimplementing the
              interface for existing systems. In such a situation, "make data entry of
              existing systems easier" might still have been a contender. In a
              situation like this, rather than embark on a 3 year integration project
              and hoping that you chose right, you might start out with a 3 month
              spike. For example you might pick two highly used systems that also have
              highly redundant data, and try to integrate parts of their databases
              together. Then you would deploy that into operation and evaluate its
              impact.

              This would yield valuable information to help you decide whether or not
              system integration is indeed the right choice. You might find out that
              the cost of integration is lower than you expected, or on the contrary,
              higher. Or you might find that integrating data from those two systems
              did not really decrease data entry time that much because the old
              practice of copying and pasting from different systems was relatively
              efficient after all.

              After that 3 month spike, you might decide that system integration is
              still the way to go and embark on another 3 month release plan in that
              direction. Or, you might decide that "make data entry easier" is
              starting too look more and more attractive and instead try a 3 month
              spike in that direction. For example, you might try to put links or
              buttons in one system, that lead you to appropriate places in other
              systems, to make copying and pasting of data more efficient. This too
              would yield valuable information. You might find that in the same 3
              months, this approach allowed you to decrease data entry time for 5
              different systems as opposed to the the 2 systems that you tried to
              integrate in the first spike. Or you might find that this "kludge"
              simply does not help with data entry time, and that you should indeed
              return to the systems integration approach. Or you might find that what
              makes the most sense is to use the system integration approach for the
              most tightly coupled (in terms of shared data) and most frequently used
              systems, and use the linking kludge for all others.

              In my experience, this kind of information is very hard to get upfront
              without having had a chance to learn from the experience of deploying
              running software in the production environment.

              As a researcher working on innovative new products like: "a system that
              records the audio of meetings and allows participants to browse and
              search it after the fact to refresh their memory about what was said", I
              find the hardest part of my work is indeed, choosing the right problem
              to solve. I used to agonize for months over this kind of decision,
              talking to lots and lots of people, floating white papers around, etc...
              And in spite of this, I still got it wrong and only found out after
              sinking a year or more into solving an irrelevant problem. Now, I still
              spend a lot of time talking to people and floating white papers (but
              less than I used to), but once I think I have a good thing, I start out
              by implementing the smallest and simplest version of the concept
              possible. Then I look for places to deploy it and get feedback.
              ----
            Your message has been successfully submitted and would be delivered to recipients shortly.