Loading ...
Sorry, an error occurred while loading the content.

just one bug's enough to make a program useless

Expand Messages
  • Phlip
    AU: I have always wondered why everyone raves about Photoshop, and I have always gotten by with ImageMagick and GIMP. I got a new graphics tablet this season,
    Message 1 of 23 , Dec 27, 2005
    • 0 Attachment
      AU:

      I have always wondered why everyone raves about Photoshop, and I have
      always gotten by with ImageMagick and GIMP.

      I got a new graphics tablet this season, and it immediately brought
      out a bunch of bugs in GIMP. But I'm certainly not here to complain
      about Free Software!

      I borrowed an old copy of Photoshop, and discovered it manages my
      tablet's pressure sensitivity beautifully. Put another way, Wacom
      tuned their tablet using Photoshop, to make sure its performance
      profile would exactly match their input range.

      And I can't use this Photoshop (unless I, uh, spring for the latest
      version). The reason is Undo only undoes the last stroke. The system
      does not maintain an Undo stack. That means I can't noodle around,
      make a mistake, and then roll back to before I started noodling
      around.

      I should not use Save and Revert as a work-around, because that would
      make me put my hands on the keyboard, or skate up to the menu, too
      often, defeating much of the benefits of a graphics tablet!

      Even the market leaders can collude together in a way that allows one
      bug to make their product useless.

      How can we usabilitists catch these blind spots in design, before they
      lose us customers?

      --
      Phlip
      http://www.greencheese.org/ZeekLand <-- NOT a blog!!
    • mithil
      Hi Philip, I completely agree to you that the undo stage is restricted to one, but just to inform you there is a history pannel which records all your
      Message 2 of 23 , Dec 27, 2005
      • 0 Attachment
        Hi Philip,
         
        I completely agree to you that the undo stage is restricted to one, but just to inform you there is a history pannel which records all your activities and allows you to revert back to any stage as per your history preferences. That to is useful, check that out.
         
        Thanks and Regards,
         Mithil Y.

        Phlip <phlip2005@...> wrote:
        AU:

        I have always wondered why everyone raves about Photoshop, and I have
        always gotten by with ImageMagick and GIMP.

        I got a new graphics tablet this season, and it immediately brought
        out a bunch of bugs in GIMP. But I'm certainly not here to complain
        about Free Software!

        I borrowed an old copy of Photoshop, and discovered it manages my
        tablet's pressure sensitivity beautifully. Put another way, Wacom
        tuned their tablet using Photoshop, to make sure its performance
        profile would exactly match their input range.

        And I can't use this Photoshop (unless I, uh, spring for the latest
        version). The reason is Undo only undoes the last stroke. The system
        does not maintain an Undo stack. That means I can't noodle around,
        make a mistake, and then roll back to before I started noodling
        around.

        I should not use Save and Revert as a work-around, because that would
        make me put my hands on the keyboard, or skate up to the menu, too
        often, defeating much of the benefits of a graphics tablet!

        Even the market leaders can collude together in a way that allows one
        bug to make their product useless.

        How can we usabilitists catch these blind spots in design, before they
        lose us customers?

        --
          Phlip
          http://www.greencheese.org/ZeekLand  <-- NOT a blog!!


        Yahoo! Shopping
        Find Great Deals on Holiday Gifts at Yahoo! Shopping

      • Ron Jeffries
        ... I have no answer to that but I can suggest some neat Tablet programs. Alias Sketchbook Pro is quite neat. I also like ArtRage. Some people like OneNote; I
        Message 3 of 23 , Dec 27, 2005
        • 0 Attachment
          On Tuesday, December 27, 2005, at 8:40:16 PM, Phlip wrote:

          > How can we usabilitists catch these blind spots in design, before they
          > lose us customers?

          I have no answer to that but I can suggest some neat Tablet
          programs.

          Alias Sketchbook Pro is quite neat. I also like ArtRage.

          Some people like OneNote; I myself do not like the way it handles my
          note taking.

          Evolution is interesting as an example of a whole 'nother way of
          interacting using the tablet.

          If I had more time to program, and the inclination to write what
          turns out to be some pretty hard code, I'd be tempted to build a
          program that works the way I wish one would.

          Tablet usability is quite an interesting problem, and in general I
          think that the simpler the better.

          Ron Jeffries
          www.XProgramming.com
          Learn the principle, abide by the principle, and dissolve the principle.
          -- Bruce Lee
        • Adrian Howard
          On 28 Dec 2005, at 01:40, Phlip wrote: [snip] ... [snip] It may be because you re not PS s target market. I don t have any graphical skills - and PS, Gimp,
          Message 4 of 23 , Dec 28, 2005
          • 0 Attachment
            On 28 Dec 2005, at 01:40, Phlip wrote:
            [snip]
            > I have always wondered why everyone raves about Photoshop, and I have
            > always gotten by with ImageMagick and GIMP.
            [snip]

            It may be because you're not PS's target market.

            I don't have any graphical skills - and PS, Gimp, Paint Shop Pro,
            ImageMagick are all about as useful as each other to me :-)

            Knowing some expert PS users I know my opinion is just as valid as
            those who don't understand what all the fuss is about with Emacs or
            Eclipse because they can edit their shopping list quite happily with
            Notepad.

            [snip]
            > How can we usabilitists catch these blind spots in design, before they
            > lose us customers?
            [snip]

            There's really only two ways:

            1) Get the users to use it. Watch and listen (emphasis on the former).
            2) Learn how to think like the user thinks.

            So...

            * Lots of conversation, sometimes using lightweight tools (e.g. paper
            prototyping) and artifacts (e.g. personna)
            * Lots of hallway-usability tests.
            * Make sure that there is a good route back for usability related
            feedback as part of your regular releases
            * Make sure usability testing becomes part of the process, not
            something that's done afterwards
            * Foster UCD skills with the Customer and the Developers so it's more
            likely that issues will be spotted and addressed.
            * Foster a culture where a bad user experience is just as important
            an issue to fix as bad program design, broken functionality, etc.

            ... and stuff...

            Adrian
          • Phlip
            ... The online WebComic artists swear by it. And I doubt you d use a text editor without a complete Undo system... ... I don t understand how PS 5 passed this
            Message 5 of 23 , Dec 28, 2005
            • 0 Attachment
              Adrian Howard wrote:

              > > I have always wondered why everyone raves about Photoshop, and I have
              > > always gotten by with ImageMagick and GIMP.

              > It may be because you're not PS's target market.

              The online WebComic artists swear by it.

              And I doubt you'd use a text editor without a complete Undo system...

              > > How can we usabilitists catch these blind spots in design, before they
              > > lose us customers?
              > [snip]
              >
              > There's really only two ways:
              >
              > 1) Get the users to use it. Watch and listen (emphasis on the former).

              I don't understand how PS 5 passed this one. The point of a graphical
              art program is you can take risks and then Undo them. This is the slim
              benefit over hardware art tools, where Undo is a lot of work.

              Put another way, computers created the 3D animation industry, but they
              have not yet shown a clear advantage on 2D sketching!

              > * Foster a culture where a bad user experience is just as important
              > an issue to fix as bad program design, broken functionality, etc.

              Oh, yeah... Undo is sometimes really hard to retrofit. For a graphical
              program, you can record a snapshot of the entire canvas after each
              action. Then memory fills up.

              So if the cost of a fix is higher than some user's nuisance factor,
              the boss will just declare the program ready to ship.

              --
              Phlip
              http://www.greencheese.org/ZeekLand <-- NOT a blog!!
            • William Pietri
              ... Are you familiar with the History stack? http://www.digitalmastery.com/tips/051203tip.html William
              Message 6 of 23 , Dec 28, 2005
              • 0 Attachment
                Phlip wrote:

                >And I can't use this Photoshop (unless I, uh, spring for the latest
                >version). The reason is Undo only undoes the last stroke. The system
                >does not maintain an Undo stack. That means I can't noodle around,
                >make a mistake, and then roll back to before I started noodling
                >around.
                >
                >


                Are you familiar with the History stack?

                http://www.digitalmastery.com/tips/051203tip.html


                William
              • Phlip
                ... Thanks, guys. Allow me to be too graceful and sheepish to emit a screed about writing a complex Undo feature and not calling it Undo. ;-) -- Phlip
                Message 7 of 23 , Dec 28, 2005
                • 0 Attachment
                  > Are you familiar with the History stack?
                  >
                  > http://www.digitalmastery.com/tips/051203tip.html

                  Thanks, guys.

                  Allow me to be too graceful and sheepish to emit a screed about
                  writing a complex Undo feature and not calling it Undo. ;-)

                  --
                  Phlip
                  http://www.greencheese.org/ZeekLand <-- NOT a blog!!
                • Phlip
                  Here ya go: http://www.greencheese.org/ZeekLand Rat snuck into the most expensive restaurant possible, and ordered like 26 dishes. Their arrival requires full
                  Message 8 of 23 , Dec 28, 2005
                  • 0 Attachment
                    Here ya go:

                    http://www.greencheese.org/ZeekLand

                    Rat snuck into the most expensive restaurant possible, and ordered
                    like 26 dishes. Their arrival requires full color, so you can tell the
                    difference between the giant popovers and the kudzu falafel.

                    So Santa delivered a Wacom tablet (note to self - don't discuss
                    Santa's usability profile); I did most of the colors in GIMP, and
                    finished with PhotoShop just now, reassured by this thread it wasn't
                    going to force me to toast a lot of good changes after one mistake.

                    Naturally, knowing the Undo feature was there helped me not use it.
                    And for some strange reason, GIMP and PhotoShop have almost the same
                    feature set!

                    Then I switched to a non-RGB format, so the file size wouldn't be
                    insanely huge. >sigh<

                    --
                    Phlip
                    http://www.greencheese.org/ZeekLand <-- NOT a blog!!
                  • Ron Jeffries
                    ... Wow. Color makes a BIG difference. Love it! Really draws me into the story. Very interesting to see and feel the change. Note to usability self: color
                    Message 9 of 23 , Dec 29, 2005
                    • 0 Attachment
                      On Thursday, December 29, 2005, at 2:36:03 AM, Phlip wrote:

                      > http://www.greencheese.org/ZeekLand

                      > Rat snuck into the most expensive restaurant possible, and ordered
                      > like 26 dishes. Their arrival requires full color, so you can tell the
                      > difference between the giant popovers and the kudzu falafel.

                      Wow. Color makes a BIG difference. Love it! Really draws me into the
                      story. Very interesting to see and feel the change.

                      Note to usability self: color matters.

                      > Naturally, knowing the Undo feature was there helped me not use it.

                      That's cool -- and interesting. I wonder why ...

                      > And for some strange reason, GIMP and PhotoShop have almost the same
                      > feature set!

                      Surprise indeed ...

                      Ron Jeffries
                      www.XProgramming.com
                      Speculation or experimentation - which is more likely to give the correct answer?
                    • Desilets, Alain
                      How can we usabilitists catch these blind spots in design, before they lose us customers? -- Alain: Sorry for late response... Catching up on pre-Xmas break
                      Message 10 of 23 , Jan 3, 2006
                      • 0 Attachment
                        How can we usabilitists catch these blind spots in design, before they
                        lose us customers?

                        -- Alain:
                        Sorry for late response... Catching up on pre-Xmas break email.

                        By their nature, bugs cannot be caught by design. They can only be
                        caught after the buggy implementation has been implemented. This is one
                        of the great advantage of early delivery and TDD. They allow you to get
                        actual running software in the hands of users early, and catch bugs
                        early on in the process.
                        ----
                      • Jeff Patton
                        ... one ... Looking up to Phlip s original post, I was seeing the bug reported as not supporting multiple levels of undo. Then we found there was a history
                        Message 11 of 23 , Jan 3, 2006
                        • 0 Attachment
                          --- In agile-usability@yahoogroups.com, "Desilets, Alain"
                          <alain.desilets@n...> wrote:
                          >
                          > How can we usabilitists catch these blind spots in design, before they
                          > lose us customers?
                          >
                          > -- Alain:
                          > Sorry for late response... Catching up on pre-Xmas break email.
                          >
                          > By their nature, bugs cannot be caught by design. They can only be
                          > caught after the buggy implementation has been implemented. This is
                          one
                          > of the great advantage of early delivery and TDD....
                          > ----

                          Looking up to Phlip's original post, I was seeing the bug reported as
                          not supporting multiple levels of undo. Then we found there was a
                          history panel - an alternatively named feature that accomplished
                          similar results to undo. It appears all these features performed as
                          the publisher expected - so none of these were bugs. Practices like
                          TDD don't catch things like this.

                          There's a couple things going on here that caught my eye/brain:

                          I like that Phlip referred to this as a blind spot in _design_. One of
                          my pet peeves is the use of the word design to refer almost strictly to
                          the technical junk under the UI. In this case the word design refers
                          to the decision to include or exclude a feature. This is the sort
                          of /design/ work that's aften called "requirements" work in many
                          shops.

                          Others already gave answers to the "how do you catch these blind
                          spots?" question. Test. But test doesn't refer to anything automated,
                          or something a tester does to make sure a feature works as designed,
                          but rather observe users actually trying to do the stuff they want to
                          do with the application.

                          So, recapping, the two things that popped for me:
                          * design is something that occurrs before code is written as part of a
                          process to choose features. [That's no news for the UCD community,
                          possible big news to requirements engineers.]
                          * a process or methodology that's concerned with the usability of it's
                          product should include some form of usability testing - that is
                          observation of target users using the product to accomplish real work -
                          not bug detection.

                          So, if there's an agile approach that's usability friendly, it's gunna
                          have these sorts of concepts stitched in.

                          Thanks,

                          -Jeff
                        • Phlip
                          ... Alain implied early feedback from customers does. Customer: I undid twice and could not step back through edits. Team: That s not a bug it s a feature.
                          Message 12 of 23 , Jan 3, 2006
                          • 0 Attachment
                            Jeff Patton wrote:

                            > Looking up to Phlip's original post, I was seeing the bug reported as
                            > not supporting multiple levels of undo. Then we found there was a
                            > history panel - an alternatively named feature that accomplished
                            > similar results to undo. It appears all these features performed as
                            > the publisher expected - so none of these were bugs. Practices like
                            > TDD don't catch things like this.

                            Alain implied early feedback from customers does.

                            Customer: I undid twice and could not step back through edits.

                            Team: That's not a bug it's a feature. (Try the History panel.)

                            Customer: Okay, the bug is Undo isn't wired to History.

                            Boss: There's a work-around. Ship it.

                            End user: I undid twice and could not step back through edits.

                            And so on; the feedback cycle engages!

                            > * a process or methodology that's concerned with the usability of it's
                            > product should include some form of usability testing - that is
                            > observation of target users using the product to accomplish real work -
                            > not bug detection.

                            Naivete is priceless.

                            I suspect that the entire GUI industry is one big blind spot, because
                            nobody made the leap to X. I don't know what it is either, because we
                            are all familiar with windows and mice.

                            --
                            Phlip
                            http://www.greencheese.org/ZeekLand <-- NOT a blog!!
                          • Brian Marick
                            ... In shops that use manual testing, whether scripted or exploratory, the testers are the people in the shop who use the product most (by a huge margin).
                            Message 13 of 23 , Jan 3, 2006
                            • 0 Attachment
                              On Jan 3, 2006, at 11:32 AM, Jeff Patton wrote:

                              > Others already gave answers to the "how do you catch these blind
                              > spots?" question. Test. But test doesn't refer to anything
                              > automated,
                              > or something a tester does to make sure a feature works as designed,
                              > but rather observe users actually trying to do the stuff they want to
                              > do with the application.

                              In shops that use manual testing, whether scripted or exploratory,
                              the testers are the people in the shop who use the product most (by a
                              huge margin). Often, they're repeating the same workflow over and
                              over, with small variations. Inefficiencies in the workflow really
                              start to drive you crazy when you hit them again and again and again
                              - whether you're a tester or user. Opportunities for improvement leap
                              out at you.

                              Testers also tend to become expert users, who have a different usage
                              profile than novices. It seems to me that most of the people used in
                              usability testing are novices (to the new features, at least) and do
                              not have chances to become experts. But I've rarely worked on a
                              product with the formal role of experience designer.

                              I've never seen a case where tester comments on the design were
                              intentionally valued. Sometimes testers don't bother. Sometimes they
                              put the bugs into the bug tracking system, usually at the "feature
                              request" (aka "lowest") level.

                              When I've seen tester observations actually become valued, it's been
                              because the tester felt comfortable going directly to the designer.
                              (In my one experience with that, my relationship to the designer felt
                              like the relationships I see between testers and customers/product
                              owners/goal donors.)

                              Good experience design may have a better way to see what testers see,
                              but perhaps testers fit into the Big Picture of Agile UX.

                              -----
                              Brian Marick, independent consultant
                              Mostly on agile methods with a testing slant
                              www.exampler.com, www.testing.com/cgi-bin/blog
                              Book in progress: www.exampler.com/book
                            • Jeff Patton
                              ... designed, ... want to ... a ... again ... leap ... Hope I didn t come across as slamming testers. ;-) Actually, it s not the tester, but the nature of the
                              Message 14 of 23 , Jan 3, 2006
                              • 0 Attachment
                                --- In agile-usability@yahoogroups.com, Brian Marick <marick@t...>
                                wrote:
                                > On Jan 3, 2006, at 11:32 AM, Jeff Patton wrote:
                                >
                                > > Others already gave answers to the "how do you catch these blind
                                > > spots?" question. Test. But test doesn't refer to anything
                                > > automated,
                                > > or something a tester does to make sure a feature works as
                                designed,
                                > > but rather observe users actually trying to do the stuff they
                                want to
                                > > do with the application.
                                >
                                > In shops that use manual testing, whether scripted or exploratory,
                                > the testers are the people in the shop who use the product most (by
                                a
                                > huge margin). Often, they're repeating the same workflow over and
                                > over, with small variations. Inefficiencies in the workflow really
                                > start to drive you crazy when you hit them again and again and
                                again
                                > - whether you're a tester or user. Opportunities for improvement
                                leap
                                > out at you.

                                Hope I didn't come across as slamming testers. ;-) Actually, it's
                                not the tester, but the nature of the test I was getting at. You hit
                                on it here. There's a difference between testing that the feature
                                performas as specified and actually using the product as intended and
                                making an often subjective judgement about how well it works. That's
                                the kind of testing that could be done in house by testers.

                                I have observed a pattern where testers become acutely aware of
                                inefficiencies in the workflow for the test cases they execute often
                                without regard for how realistically the test case reflects an actual
                                users' goals or usage.

                                For example I used to do a lot of work in a brick and mortar retail
                                environment. Testers often asked for features to create items to
                                sell with less mandatory attributes, or navigate directly from item
                                creation to transaction entry, or to easily remove items. While all
                                these seemed logical, in large retail organizations those creating
                                items weren't the ones entering transactions. And removing items
                                with any transactional history had larger legal implications. The
                                feature suggestions would have indeed improved workflow for the
                                tester executing test cases, but not necessarily the user doing work.

                                Now I know enlightened testers such as Brian and others wouldn't fall
                                into this trap. But to help the less enlightened I've always found
                                it important to have a strong understanding of the applications users
                                and their workflow. A good user model and task model serve that
                                purpose. Some concisely written user scenarios help with that also.
                                It's hard to remember sometimes that we're not the user and the
                                models help remind us.

                                > I've never seen a case where tester comments on the design were
                                > intentionally valued. Sometimes testers don't bother. Sometimes
                                they
                                > put the bugs into the bug tracking system, usually at the "feature
                                > request" (aka "lowest") level.
                                >
                                > When I've seen tester observations actually become valued, it's
                                been
                                > because the tester felt comfortable going directly to the
                                designer.
                                > (In my one experience with that, my relationship to the designer
                                felt
                                > like the relationships I see between testers and customers/product
                                > owners/goal donors.)

                                I think it's important to suggest features in some context - by that
                                I mean some model understanding of the user that allows us to
                                determine if the feature is indeed good. A feature suggestion on its
                                own doesn't give us any context to judge how good it is. Designers
                                or product owners generally carry a model of the users and usage in
                                their head if not explicitly on paper. They're better able to judge
                                if the feature is or isn't valuable.

                                > Good experience design may have a better way to see what testers
                                see,
                                > but perhaps testers fit into the Big Picture of Agile UX.

                                Yes - I'd agree - but not all testers. This "testing for optimizing
                                user experience" isn't something I've observed all testers agree
                                with, even in an agile context. That's the kind of testing I'd like
                                to see done. Specifically who does it and how is the bit to be
                                resolved in the organization doing it.

                                It's interesting also that this isn't the type of test that can be
                                automated.
                              • Desilets, Alain
                                ... one ... Looking up to Phlip s original post, I was seeing the bug reported as not supporting multiple levels of undo. Then we found there was a history
                                Message 15 of 23 , Jan 3, 2006
                                • 0 Attachment
                                  > How can we usabilitists catch these blind spots in design, before they

                                  > lose us customers?
                                  >
                                  > -- Alain:
                                  > Sorry for late response... Catching up on pre-Xmas break email.
                                  >
                                  > By their nature, bugs cannot be caught by design. They can only be
                                  > caught after the buggy implementation has been implemented. This is
                                  one
                                  > of the great advantage of early delivery and TDD....
                                  > ----

                                  Looking up to Phlip's original post, I was seeing the bug reported as
                                  not supporting multiple levels of undo. Then we found there was a
                                  history panel - an alternatively named feature that accomplished
                                  similar results to undo. It appears all these features performed as
                                  the publisher expected - so none of these were bugs. Practices like
                                  TDD don't catch things like this.

                                  -- Alain:
                                  Given the title of Phlip's posting, I was writing with the assumption
                                  that the reported problem was a bug (i.e. developpers did implement a
                                  multi-level undo feature but it somehow did not work on Phlip's machine)
                                  as opposed to a design problem (i.e. feature deemed not necessary when
                                  it was in fact essential, or feature exists, but is hidden).

                                  So I stand by what I said. If the problem is a bug, no amount of design
                                  related work will find it.

                                  In this particular case, it turns out what Phlip thought was a bug was
                                  in fact a problem with bad design... The feature existed but it was
                                  hidden too well. But I have seen cases where a bug makes an otherwise
                                  well designed software completely unusable.

                                  The point is this. If you want your software to be usable, you have to
                                  pay attention to the design, but also make sure that the implementation
                                  works correctly. And the only way to identify cases where it does not is
                                  to test the actual implementation continuously through TDD, and also,
                                  put it in the hands of real test users as quickly as possible.
                                  ----
                                • Desilets, Alain
                                  -- Alain: You forgot two important and very likely interventions in the conversation below. I insert them below. ... Customer: I undid twice and could not step
                                  Message 16 of 23 , Jan 3, 2006
                                  • 0 Attachment
                                    -- Alain:
                                    You forgot two important and very likely interventions in the
                                    conversation below. I insert them below.
                                    ---

                                    Customer: I undid twice and could not step back through edits.

                                    Team: That's not a bug it's a feature. (Try the History panel.)

                                    Customer: Okay, the bug is Undo isn't wired to History.

                                    Boss: There's a work-around. Ship it.

                                    -- Alain:
                                    At this point, the developper would probably say:

                                    Developper: Wait a minute... Wiring Ctrl-Z to History is just 1 hour of
                                    work on my part. And that includes testing, writing unit tests, and
                                    integrating into the main version. I could do it in my sleep!

                                    Boss: OK then, make it so.
                                    -----
                                  • Desilets, Alain
                                    In shops that use manual testing, whether scripted or exploratory, the testers are the people in the shop who use the product most (by a huge margin). Often,
                                    Message 17 of 23 , Jan 3, 2006
                                    • 0 Attachment
                                      In shops that use manual testing, whether scripted or exploratory,
                                      the testers are the people in the shop who use the product most (by a
                                      huge margin). Often, they're repeating the same workflow over and
                                      over, with small variations. Inefficiencies in the workflow really
                                      start to drive you crazy when you hit them again and again and again
                                      - whether you're a tester or user. Opportunities for improvement leap
                                      out at you.

                                      -- Alain:
                                      I agree with this. I write a lot of unit tests, but I do a lot of manual
                                      testing on top of it. And through that, I get a lot of excellent
                                      insights into where the usability hotspots are, and not only for expert
                                      users. Like you said, when you use the software over and over again,
                                      even the lightest pain starts jumping at you.

                                      Of course, I don't rely just on that for assessing usability of the
                                      system. I also pay a lot of attention to customer feedback. Everytime
                                      someone asks for help is treated as an occasion to get insights into
                                      usability problems.
                                      ----
                                    • Desilets, Alain
                                      Hope I didn t come across as slamming testers. ;-) Actually, it s not the tester, but the nature of the test I was getting at. You hit on it here. There s a
                                      Message 18 of 23 , Jan 3, 2006
                                      • 0 Attachment
                                        Hope I didn't come across as slamming testers. ;-) Actually, it's
                                        not the tester, but the nature of the test I was getting at. You hit
                                        on it here. There's a difference between testing that the feature
                                        performas as specified and actually using the product as intended and
                                        making an often subjective judgement about how well it works. That's
                                        the kind of testing that could be done in house by testers.

                                        I have observed a pattern where testers become acutely aware of
                                        inefficiencies in the workflow for the test cases they execute often
                                        without regard for how realistically the test case reflects an actual
                                        users' goals or usage.

                                        -- Alain:
                                        I for one, never use manual to repeatively test the same scenario.
                                        That's something better reserved for machines (i.e. done through
                                        automated unit testing).

                                        When I talked about manual testing (in my response to Bryan's posting),
                                        I was talking about exploratory testing, which, as I practice it
                                        involves two things:

                                        - trying to come up with creative ways to destroy the system
                                        - using the system to carry out some realistic task (i.e. a task that a
                                        a real user might do)

                                        The second kind of testing does allow me to spot a lot of usability
                                        problems at the design level, but also bugs that stand in the way of
                                        usability.
                                        ----
                                      • Phlip
                                        ... Maybe PhotoShop defended the feature where Undo-Undo redoes the undid thing. That s actually useful in some situations, because you can do blink
                                        Message 19 of 23 , Jan 3, 2006
                                        • 0 Attachment
                                          Desilets, Alain wrote:

                                          > Developper: Wait a minute... Wiring Ctrl-Z to History is just 1 hour of
                                          > work on my part. And that includes testing, writing unit tests, and
                                          > integrating into the main version. I could do it in my sleep!

                                          Maybe PhotoShop defended the "feature" where Undo-Undo redoes the undid thing.

                                          That's actually useful in some situations, because you can do "blink
                                          comparator" from one keystroke.

                                          Never mind.

                                          --
                                          Emily Litella
                                        • Brian Marick
                                          ... Agreed. Going beyond what you say: a tester without understanding of the domain will not only make bad usability suggestions, she will also report too many
                                          Message 20 of 23 , Jan 3, 2006
                                          • 0 Attachment
                                            On Jan 3, 2006, at 2:18 PM, Jeff Patton wrote:
                                            >
                                            > I have observed a pattern where testers become acutely aware of
                                            > inefficiencies in the workflow for the test cases they execute often
                                            > without regard for how realistically the test case reflects an actual
                                            > users' goals or usage.
                                            >
                                            > For example I used to do a lot of work in a brick and mortar retail
                                            > environment. Testers often asked for features to create items to
                                            > sell with less mandatory attributes, or navigate directly from item
                                            > creation to transaction entry, or to easily remove items. While all
                                            > these seemed logical, in large retail organizations those creating
                                            > items weren't the ones entering transactions. And removing items
                                            > with any transactional history had larger legal implications. The
                                            > feature suggestions would have indeed improved workflow for the
                                            > tester executing test cases, but not necessarily the user doing work.
                                            >
                                            > Now I know enlightened testers such as Brian and others wouldn't fall
                                            > into this trap. But to help the less enlightened I've always found
                                            > it important to have a strong understanding of the applications users
                                            > and their workflow. A good user model and task model serve that
                                            > purpose. Some concisely written user scenarios help with that also.
                                            > It's hard to remember sometimes that we're not the user and the
                                            > models help remind us.

                                            Agreed. Going beyond what you say: a tester without understanding of
                                            the domain will not only make bad usability suggestions, she will
                                            also report too many bugs that don't matter to real users and too few
                                            bugs that do.

                                            As Michael Bolton and a host of others would instantly remind me, the
                                            good tester is also operating under the assumption that the good user
                                            model and good task model are wrong in some important ways, and that
                                            part of her job is probably to find out how. (Similarly, her own
                                            understanding is doubtless wrong, too. It's the Turtles of
                                            Uncertainty all the way down.)


                                            P.S. Making changes to improve the usability for testers does have
                                            business value, unless their time costs nothing. I have some faith,
                                            but no evidence, that it would also improve the code, much like
                                            catering to JUnit or Fit does.

                                            -----
                                            Brian Marick, independent consultant
                                            Mostly on agile methods with a testing slant
                                            www.exampler.com, www.testing.com/cgi-bin/blog
                                            Book in progress: www.exampler.com/book
                                          • Desilets, Alain
                                            Maybe PhotoShop defended the feature where Undo-Undo redoes the undid thing. That s actually useful in some situations, because you can do blink comparator
                                            Message 21 of 23 , Jan 3, 2006
                                            • 0 Attachment
                                              Maybe PhotoShop defended the "feature" where Undo-Undo redoes the undid
                                              thing.

                                              That's actually useful in some situations, because you can do "blink
                                              comparator" from one keystroke.

                                              Never mind.

                                              -- Alain:
                                              Interesting point. Do you think this is something that you would be
                                              likely to find out through upfront design and paper prototyping?

                                              My guess is that this is exactly the kind of thing that does not become
                                              apparent until (even to the end user) until you look at real users
                                              laying their hands on the real thing and trying to carry out a real
                                              task.
                                              ----
                                            • Jeff Patton
                                              ... In the situation I was thinking about, making changes to the delivered product to better support testers would have broken business rules in the
                                              Message 22 of 23 , Jan 3, 2006
                                              • 0 Attachment
                                                --- In agile-usability@yahoogroups.com, Brian Marick <marick@t...>
                                                wrote:
                                                > P.S. Making changes to improve the usability for testers does have
                                                > business value, unless their time costs nothing. I have some faith,
                                                > but no evidence, that it would also improve the code, much like
                                                > catering to JUnit or Fit does.

                                                In the situation I was thinking about, making changes to the delivered
                                                product to better support testers would have broken business rules in
                                                the application. Basically it would have made it easier/faster for the
                                                tester to set up and tear down test data by forgoing some of the rules
                                                around creating and deleting - and also installing quick navigation to
                                                jump from creation directly to transaction entry, again, inapropriate
                                                for our app, but workflow the testers did often.

                                                Alain is right that lots of these tedious bits of testing should be
                                                automated.

                                                You're right that spending a little money to help testers complete
                                                their work faster - or not go crazy doing it - is a good idea. We just
                                                need to know why we're doing it. If it was a feature strictly for
                                                testing, I'd want some way to hide or disable it from a production
                                                release.

                                                But, your point is well taken. Also Michael's point you referred to as
                                                well.

                                                thanks,

                                                -Jeff
                                              • Larry Constantine
                                                ... Many bugs (both code bugs and usability defects) can be avoided by effective design practices and found prior to having running software through
                                                Message 23 of 23 , Jan 4, 2006
                                                • 0 Attachment
                                                  Alain wrote:

                                                  > By their nature, bugs cannot be caught by design. They can only be
                                                  > caught after the buggy implementation has been implemented. This is one
                                                  > of the great advantage of early delivery and TDD. They allow you to get
                                                  > actual running software in the hands of users early, and catch bugs
                                                  > early on in the process.

                                                  Many bugs (both code bugs and usability defects) can be avoided by effective
                                                  design practices and found prior to having running software through
                                                  inspections and walkthroughs. Multiple research studies and extensive
                                                  practice have shown inspections to be more cost effective than is testing
                                                  for finding and eliminating bugs (both kinds).

                                                  --Larry Constantine, IDSA [mailto:lconstantine@...]
                                                    Chief Scientist | Constantine & Lockwood, Ltd.

                                                  > -----Original Message-----
                                                  > From: agile-usability@yahoogroups.com
                                                  [mailto:agile-usability@yahoogroups.com] On Behalf
                                                  > Of Desilets, Alain
                                                  > Sent: Tuesday, 03 January 2006 11:40 AM
                                                  > To: agile-usability@yahoogroups.com
                                                  > Subject: RE: [agile-usability] just one bug's enough to make a program
                                                  useless
                                                  >
                                                  > How can we usabilitists catch these blind spots in design, before they
                                                  > lose us customers?
                                                  >
                                                  > -- Alain:
                                                  > Sorry for late response... Catching up on pre-Xmas break email.
                                                  >
                                                  > ----
                                                  >
                                                  >
                                                  >
                                                  > Yahoo! Groups Links
                                                  >
                                                  >
                                                  >
                                                  >
                                                Your message has been successfully submitted and would be delivered to recipients shortly.