Loading ...
Sorry, an error occurred while loading the content.
 

RE: [agile-testing] Requirement Traceability Matrix and Agile Testing

Expand Messages
  • Michael Bolton
    Michael: What s the difference between traceability and a traceability document ? Todd: The first is a data structure, and the second is a human-readable
    Message 1 of 16 , Mar 14, 2008
      Michael:>> What's the difference between "traceability" and "a traceability
      document"?

      Todd:>The first is a data structure, and the second is a human-readable
      version of the data structure. Or maybe I don't understand the point of
      your question. Is there a point you're trying to make?

      That's an interesting answer. I don't think of traceability as a data
      structure as such. I think of it as something more abstract--a concept that
      helps us understand the historical links between one set of ideas and
      another, or an answer to a question in the general form of "Why did we do
      that?" More extremely, it might be "Who did X, and what did they do, and
      why, or when, or how did they do it?" Answers to those questions can take
      many forms, depending on the culture. Conversations, stories, strategic
      themes, histories, logs, journals, source code, automated tests, design
      documents, daily scrums, email, and (yes) spreadsheets are all media in
      which traceability might be found, and all of them can be passed from one
      person to another. But only some of these forms are documents that are
      explicitly designed for that purpose and for no other. I'd suggest that you
      need traceability if there's a danger of losing corporate memory that is
      genuinely valuable--but what's the risk of that when it's already available
      in so many forms?

      Now, there are plausibly legitimate answers to a question like this. The
      trouble (and clutter, and extra expense, and extra overhead) tends to come
      when we haven't asked the question at all. That would be the suggestion
      that I would make to the Original Poster: Someone wants traceability? Ask
      what they want it for, how often they'll want it, the form in which they'll
      want it, the forms in which they might be able to accept it--and then ask if
      the cost will be matched by value. Things that can be read by a human
      typically require a human to write them--yet most projects I've seen tend to
      be short on extra humans.

      Recently I saw Alton Brown, the celebrity cook, on TV. He demonstrated a
      gizmo that's designed to take the smell of garlic off your hands. It's
      essentially a lump of stainless steel, and there's something about the
      chemistry that removes the garlic. Cool idea. But he chucked it, saying
      that you can get exactly the same effect by rubbing your hands on the flat
      of your stainless steel chef's knife (or if you have one of those knives
      with a stainless steel handle, you can use that). Why buy an extra
      tchotchke to clutter up the kitchen when you've already got what you need?

      Another variation on Mike Emeigh's good suggestion of the Five Whys, by the
      way, is a set of questions that I first heard from Don Gray: 1) What will
      happen if I do? 2) What won't happen if I do? 3) What will happen if I
      don't? 4) What won't happen if I don't?

      ---Michael B.
    • Ron Jeffries
      ... What prevents you from running the tests and seeing which ones break? What prevents you from naming the tests after the corresponding requirements stories?
      Message 2 of 16 , Mar 14, 2008
        Hello, Todd. On Friday, March 14, 2008, at 3:26:52 PM, you wrote:

        >> Hello, Todd. On Friday, March 14, 2008, at 1:55:20 PM, you wrote:
        >>
        >> > But requirements for a
        >> > feature sometimes change in unpredictable ways later. So
        >> I'm always a
        >> > little nervous that the test I ran 6 months ago doesn't
        >> really fully
        >> > cover the requirement as it exists today.
        >>
        >> What prevents you from writing new tests when requirements change?

        > If you don't have some sort of link between requirements and tests, how
        > do you know which tests are "dirty" and need to be updated due to a
        > requirements change? That link is traceability.

        What prevents you from running the tests and seeing which ones
        break?

        What prevents you from naming the tests after the corresponding
        requirements stories?

        Ron Jeffries
        www.XProgramming.com
        If not now, when? -- The Talmud
      • Doug Swartz
        ... This is essentially what we do. We write new test scenarios to verify that the new requirement is working correctly. Then we run the pre-existing tests.
        Message 3 of 16 , Mar 14, 2008
          Friday, March 14, 2008, 4:02:06 PM, Ron Jeffries wrote:

          > Hello, Todd. On Friday, March 14, 2008, at 3:26:52 PM, you wrote:

          >>> Hello, Todd. On Friday, March 14, 2008, at 1:55:20 PM, you wrote:
          >>>
          >>> > But requirements for a
          >>> > feature sometimes change in unpredictable ways later. So
          >>> I'm always a
          >>> > little nervous that the test I ran 6 months ago doesn't
          >>> really fully
          >>> > cover the requirement as it exists today.
          >>>
          >>> What prevents you from writing new tests when requirements change?

          >> If you don't have some sort of link between requirements and tests, how
          >> do you know which tests are "dirty" and need to be updated due to a
          >> requirements change? That link is traceability.

          > What prevents you from running the tests and seeing which ones
          > break?

          This is essentially what we do. We write new test scenarios to
          verify that the new requirement is working correctly. Then we
          run the pre-existing tests. When any of them don't pass, we
          examine the failure to see if we broke something we shouldn't
          have; or if the old test needs to be updated to reflect the
          changed requirements.

          Perhaps it seems too simple to work, but it has, so far.


          --

          Doug Swartz
          daswartz@...
        • John Overbaugh
          In the spirit of being agile... Who writes your RTVM and how is it/are they stored? Seems to me one of the liberating benefits of Agile is a reduction of
          Message 4 of 16 , Mar 14, 2008
            In the spirit of being agile... Who writes your RTVM and how is it/are they stored? Seems to me one of the liberating benefits of Agile is a reduction of documentation - wherever it results in improved quality/decreased effort. In my role as TM for a major retailer's merchant system transformation project, we wasted months of time 1) reading technical and functional specs, 2) entering RTVM into one system, 3) exporting RTVM and importing into Quality Center and 4) mapping test cases.

            Why can't the RTVM be THE test cases, or at least a sub-set of the cases? Why can't RTVM be entered as part of a user story (for instance, as acceptance tests or the likes)? Why can't test cases plus a deep test case review replace RTVM entirely?

            Our system was imposed by years of practice as well as the presence of a certain, uh, blue partner. I was making progress - we did move to eliminate more than one of the four steps above, but habit is tough to break and it took a lot of time. I see the 'value' in that RTVM allows mapping of independent validation cases back to functional requirements. I think, though, that the whole concept of RTVM screams distrust of the testing organization. Having seen many test orgs, I can say that there are situations where that distrust is valid. There are, however, many more situations where the test org is just as committed and intrested in a quality release as the business is--in those circumstances, trusting the test org is one of the best things that can be done.

            Oops - sorry, I got up on my soap box for a bit there!

            Anyhow, there must be a way to eliminate the silos and just strive for efficient, effective test coverage.

            John O.
            http://thoughtsonqa.blogspot.com

            On Fri, Mar 14, 2008 at 11:55 AM, Bradley, Todd <todd.bradley@...> wrote:

            If you're talking about a traceability matrix as being a document that
            shows connections between requirements and tests, I don't currently have
            one. But I'd like to put one together. We develop tests in each
            sprint, as functionality is being developed. But requirements for a
            feature sometimes change in unpredictable ways later. So I'm always a
            little nervous that the test I ran 6 months ago doesn't really fully
            cover the requirement as it exists today.

            Todd.



            > -----Original Message-----
            > From: agile-testing@yahoogroups.com
            > [mailto:agile-testing@yahoogroups.com] On Behalf Of Jorge Argus
            > Sent: Friday, March 14, 2008 8:26 AM
            > To: agile-testing@yahoogroups.com
            > Subject: [agile-testing] Requirement Traceability Matrix and
            > Agile Testing
            >
            > Since I'm still fairly new to agile - I was just asked about
            > adding a traceability matrix. Has anyone used a traceability
            > matrix with agile testing? My first response was that the
            > team didn't need this - since it's built into our (company
            > mangled) FIT, where everything is being covered. Has anyone
            > else faced this question? Does anyone use a secret RTM on the
            > side? Thanks.
            >
            >
            > ------------------------------------
            >
            > Yahoo! Groups Links
            >
            >
            >
            >
            >



            --
            John Overbaugh
            blog: http://thougthsonqa.blogspot.com
            site: http://www.yourtestmanager.com
            <<john at your test manager dot com>>
          • Brad Appleton
            ... Right on Michael! The difference between traceability and a traceability *document* or *matrix* is ... Traceability is a desired property of a project,
            Message 5 of 16 , Mar 15, 2008
              Michael Bolton wrote:
              > That's an interesting answer. I don't think of traceability as a data
              > structure as such. I think of it as something more abstract--a concept that
              > helps us understand the historical links between one set of ideas and
              > another, or an answer to a question in the general form of "Why did we do
              > that?"

              Right on Michael!

              The difference between "traceability" and a traceability *document* or
              *matrix* is ...

              "Traceability" is a desired property of a project, one that makes a
              statement about the transparency and visibility of the "important"
              pieces of information & artifacts and the ability to see and navigate
              their interrelationships.

              A traceability "document" or "matrix" is one presumed implementation of
              providing this "desired property" (and a traceability matrix is not a
              particularly agile way of doing it, assuming it has to be manually
              created & maintained).

              Manually created & updated/maintained traceability matrices and
              documents are one of the most tedious, time-consuming, and tiresome ways
              of "demonstrating" traceability. You want to find an easier way if at
              all possible, one which is more or less automatics, or better yet, is
              simply transparent as a result of how you do your work. Hence my mantra:

              "Trustworthy Transparency /over/ Tiresome Traceability"

              The bottom-line is that the extent of traceability you genuinely require
              may be different from that which many might automatically assume. And
              more importantly, for whatever degree of traceability you DO implement,
              you want the corresponding traceability document/report to be something
              comes almost automatically as a by-product of the transparency of your
              process, or that you can at least auto-generate without a lot of effort
              to update/maintain or to initially create.

              For a more thorough "dissection" of what the presumed purpose and value
              of traceability is, I recommend the article:
              * "The Trouble with Tracing: Traceability Dissected"
              <<http://www.cmcrossroads.com/content/view/6685/202/>>

              To read about some lean/agile approaches to attaining traceability, take
              a peek at:
              * "Lean Traceability: a smattering of strategies and solutions"
              <<http://www.cmcrossroads.com/content/view/9089/120>>

              For what I consider the XP "take" on traceability, see:
              * "eXtreme Traceability"
              <<http://blog.bradapp.net/2006/04/extreme-traceability.html>>

              (Other "Agile SCM" articles are at http://cmwiki.com/AgileSCMArticles)
              --
              Brad Appleton <brad {AT} bradapp.net>
              Agile CM Environments (http://blog.bradapp.net/)
              & Software CM Patterns (www.scmpatterns.com)
              "And miles to go before I sleep" -- Robert Frost
            • Bradley, Todd
              ... to ... Due to a sad lack of automated test tool support for our chosen GUI technology, nearly all our tests are manual. So, I guess the answer to your
              Message 6 of 16 , Mar 17, 2008
                Ron Jeffries wrote:
                > > If you don't have some sort of link between requirements and tests,
                > > how do you know which tests are "dirty" and need to be updated due
                to
                > > a requirements change? That link is traceability.
                >
                > What prevents you from running the tests and seeing which ones break?

                Due to a sad lack of automated test tool support for our chosen
                GUI technology, nearly all our tests are manual. So, I guess the
                answer to your question is "time".

                > What prevents you from naming the tests after the
                > corresponding requirements stories?

                That's all well and good until someone comes along and renames
                the requirement. There's got to be a better link between the
                requirements and the tests than just having the same name.


                Michael Bolton wrote:
                > Michael:>> What's the difference between "traceability" and
                > "a traceability document"?
                >
                > Todd:>The first is a data structure, and the second is a
                > human-readable version of the data structure. Or maybe I
                > don't understand the point of your question. Is there a point
                > you're trying to make?
                >
                > That's an interesting answer. I don't think of traceability
                > as a data structure as such. I think of it as something more
                > abstract--a concept that helps us understand the historical
                > links between one set of ideas and another, or an answer to a
                > question in the general form of "Why did we do that?"

                You're right. "Traceability" is a property. A "trace" is
                a data structure. It is something like a doubly-linked list
                between requirements and test cases, except that every test
                can support multiple requirements, and every requirement can
                be covered by multiple tests. I can't think of what such a
                data structure is called at the moment.


                Cheers,
                Todd.
              • Bradley, Todd
                ... Yes! That s what I was trying to get at. The test cases should contain the traceability to the require- ments in some way. And the traceability matrix
                Message 7 of 16 , Mar 17, 2008
                  John O wrote:
                  > Why can't the RTVM be THE test cases, or at least a
                  > sub-set of the cases?

                  Yes! That's what I was trying to get at. The test
                  cases should contain the traceability to the require-
                  ments in some way. And the traceability matrix is
                  the human readable document generated from that data.


                  Todd.
                Your message has been successfully submitted and would be delivered to recipients shortly.