Loading ...
Sorry, an error occurred while loading the content.
 

Re: Keeping cross-cutting tasks visible (without technical stories)

Expand Messages
  • peterskeide
    Personally, I don t mind mixing and matching different types of requirement formats if the situation calls for it. User Stories are fine, and often the
    Message 1 of 28 , Dec 1, 2010
      Personally, I don't mind mixing and matching different types of requirement formats if the situation calls for it. User Stories are fine, and often the preferred format, but (imo) sometimes a simple technical requirement can be the right thing. It all depends on context. Some technical issues can easily be baked into a User Story. Some can't. If it feels wrong, it may simply be wrong.

      Perhaps a sprint backlog where the majority of issues are value focused User Stories plus a few technical requirements will work for you? Try it. If it doesn't work, try something else.

      Whatever you do, I think it pays to be open about it when talking with the Product Owner. If your team feels there is a need to work on something with a purely technical focus, explain the situation to the PO. He/she may understand, and allow you to work on it during a Sprint. If the PO doesn't allow this, and you end up having to "hide" some of these issues in unrelated User Stories, it may be a sign of other, more serious problems.

      Granted, if your team does a good job of focusing on quality when implementing User Stories, these types of issues should be rare.

      --- In scrumdevelopment@yahoogroups.com, Paul Tevis <ptevis@...> wrote:
      >
      > The team I'm on right now is about four months into a Scrum
      > transition, and we're finally starting to deal with technical debt
      > reduction and technical practice improvement. One thing that I'm
      > struggling with is how to make the work that we're doing on these
      > visible to the team, while at the same time keeping our focus on
      > delivering customer value (i.e. doing real stories).
      >
      > For example, a while back we set up a minimal Continuous Integration
      > server to make sure we didn't break the build. This was in parallel to
      > the existing system that creates the installers we send to customers,
      > which only built once a day, didn't run any tests, and was not
      > well-understood by any current members of the team. (There had been a
      > lot of turnover on the project prior to our transition.) Now, we'd
      > like to move that functionality to the CI system, to make it easier to
      > maintain, to improve our testing process, and to move closer to
      > Continuous Delivery. We know this will take a non-trivial amount of
      > work, so I'd like to keep it visible, but I don't want to make a
      > technical story for it.
      >
      > Similarly, we recently needed to cut a release branch, which involved
      > updating some scripts, etc. This was a purely technical task, one that
      > the team felt was necessary to maintain a level of quality, but that
      > wasn't connected directly to any single user story.
      >
      > My concerns are (1) as soon as we start tracking non-story tasks we'll
      > lose focus on delivering customer value, and (2) if we don't make
      > these sorts of tasks visible, we won't make progress on them at the
      > rate we need to. What are good patterns you've seen for dealing with
      > technical tasks that aren't directly attached to a story (or that cut
      > across multiple stories)?
      >
      > Thanks,
      >
      > --Paul
      >
      > --
      > Paul Tevis
      > ptevis@...
      > http://paultevis.com
      >
    • Roy Morien
      I am grieved that you are troubled , Ron. But I don t see the point in being a slave to a method ... certainly Scrum, and XP et al have great points of process
      Message 2 of 28 , Dec 1, 2010
        I am grieved that you are troubled , Ron. But I don't see the point in being a slave to a method ... certainly Scrum, and XP et al have great points of process and procedure that I have adopted ... and, as is the Scrum way, sometimes have adapted.

        But, I do respect your knowledge, and I will read your words carefully.

        But, I think my advice is sometimes worthy, being based on a number of decades of experience in the area ... going back to the last century, I must admit ... or maybe not admit, but claim with some pride.

        Regards,
        Roy Morien


        To: scrumdevelopment@yahoogroups.com
        From: ronjeffries@...
        Date: Wed, 1 Dec 2010 07:16:10 -0500
        Subject: Re: [scrumdevelopment] Keeping cross-cutting tasks visible (without technical stories)

         
        Hello, Roy. On Wednesday, December 1, 2010, at 3:29:27 AM, you
        wrote:

        > I think this notion of only ever doing things that have value to
        > the customer has been somewhat misunderstood, or misapplied. There
        > are most certainly things that need to be done that do not have an
        > immediate, direct, or even obvious value to a customer ... but
        > they still must be done. So what if user stories are not written
        > for them, and so what if they really don't fit the notion of user
        > stories (however defined). Being afraid to get it done because
        > Scrum doesn't seem to cover that situation just leads to
        > development paralysis.

        If one is to do Scrum well, one needs to understand Scrum well.
        Deviating from it every time something "seems" odd will not help a
        team to prosper. It troubles me that so much of your advice, Roy,
        deviates substantially from what Scrum and Agile teach, even when
        there are perfectly good Scrum / Agile answers.

        So here's so what.

        The fundamental point of Scrum is for the PO to prioritize work
        for the team, so that the best possible product can be delivered
        within the desired time and budget. The PO will get the best
        product if work is chosen that is of high "business value". (Low
        development cost is also a concern in choosing the work, but may
        will be substantially less important than value.)

        Clearly, the PO is capable of driving out more value when a larger
        percentage of the team's time comes under her control, since
        otherwise, she is directing a smaller flow of work, which will
        inherently reduce total value. Therefore it is undesirable to
        slice away a strip of the team's development time and make it
        unavailable for stories.

        However, so-called "technical stories" have a value proposition
        which is almost completely different from an ordinary story. A
        technical refactoring story may promise faster velocity sometime
        in the future, while an ordinary story is something that real
        users actually want. It is therefore very difficult, perhaps
        impossible, to compare an ordinary story and a technical story for
        value. Therefore it is undesirable to present technical stories at
        all, since they are so difficult to prioritize.

        This seems, if one doesn't think about it very hard, to be a
        dilemma. We can't take away the PO's time, we can't prioritize the
        technical stories. We can't have them at all. We are doomed.

        Like most dilemmas, this one is solved by using a third
        alternative. It turns out that [almost?] every bit of technical
        work, if it should be done at all, is done in support of one or
        more stories. IN PARTICULAR, the kind of work the OP is talking
        about, creation of tests and creation of clean code, is inherently
        part of creating stories.

        Why inherently, one might ask. Because for code to be DONE, it
        must work, and to know whether it works, we must test it. Because
        we are working incrementally, we will revisit much of the code
        many times and we might break it, so it is wise that the tests be
        recorded for reuse, i.e. automated.

        Further, since we are building the system incrementally, we are
        building the design incrementally. We do not have the time (nor,
        likely, the ability) to define the design correctly up front. We
        do not have the time to build it up front even if we did have the
        ability to define it. We must, perforce, evolve the design.
        Therefore, for each feature to be DONE, its design must be fit for
        the current purpose.

        Therefore, testing, and design improvement, are /inherently/ part
        of the work of doing stories.

        How does this help us with our dilemma? We have, through our human
        failings, fallen behind on the testing and design. What are we to
        do? We do not want to slice time away from the Product Owner. The
        Product Owner cannot correctly prioritize these strange technical
        stories. We are doomed.

        No. All we need to do is to bear down a bit on improving the code
        we work on, doing all it takes to bring it back up to the proper
        definition of DONE, namely "well-tested and well-factored". We
        make the code a bit better every time we pass through it.

        At first, maybe this takes a little longer per story. This is not
        a real concern, as no one but the team really knows if the team
        can do twelve this Sprint or only ten. The team commits to what
        they can do. They do it, and the world becomes a better place.

        Better yet, soon, surprisingly soon, it doesn't take longer after
        all. Since we are improving the code and tests in the areas we
        work in, velocity picks up quite quickly in the areas we work in.
        We wind up going faster than before, because we are going better.

        Our Product Owner retains her ability to prioritize the whole of
        our work, and our code improves. No confusion, no political
        discussion, no need to make promises about the future that we may
        not be able to keep.

        We don't need technical stories or tricky negotiations. We just do
        the job we were always supposed to do, namely make each story
        DONE, fully tested, with a good design in place.

        Ron Jeffries
        www.XProgramming.com
        Thousands of years ago, the first man discovered how to make fire.
        He was probably burned at the stake he had taught his brothers to
        light - Howard Roark (The Fountainhead, Ayn Rand)


      • Malcolm Anderson
        Ron This issue in Scrum of how to keep up the technical maintenance has always bothered me. I was cruising along with your answer, singing praises to your
        Message 3 of 28 , Dec 1, 2010
          Ron

          This issue in Scrum of how to keep up the technical maintenance has always bothered me.

          I was cruising along with your answer, singing praises to your ability to make the unclear, clear. 
          Until you got to the point where you seem to encourage a lack of transparency.

          >At first, maybe this takes a little longer per story. This is not a real concern, as no one but the team really knows if the team
          >can do twelve this Sprint or only ten. The team commits to what they can do. They do it, and the world becomes a better place.

          This sounds like "lying to your product owner" (and the product owners that I've worked with, do know that "the team can do twelve")
          This goes against the value of transparency. 
          It also takes away the PO's right to incur short term technical debt in order to make a particular deadline.


          One of the weaknesses of Scrum is that we think, it's self evident that product owners are responsible for maintaining the health their product while they provide value to their customer.

          I believe that this all comes down to Stewardship, and not making it clear to the product owner that they need to be Stewards to their product.

          I've got 3 questions, the answers to which I believe are currently "No, No and No"

          1) Is there a Product Owner body where Product Owners discuss with other product owners how to be a better product owner?

          2) Is the concept of Stewardship a part of the PO training?

          3) Has the PO body taking on a commitment to maintaining and improving the health of their products?


          Until the answer to the above 3 questions is Yes, Yes, and Yes, this is going to be an issue with Scrum.
          Until we have some kind of "Product Owner Oath of Stewardship," this is going to be an issue with Scrum.
          Until stewardship is self evident to people other than Scrum Masters, this is going to be an issue with Scrum.


          Thanks

          Malcolm Anderson
          Scrum Coach & Agile Engineer




          On Wed, Dec 1, 2010 at 6:16 AM, Ron Jeffries <ronjeffries@...> wrote:
           

          Hello, Roy. On Wednesday, December 1, 2010, at 3:29:27 AM, you
          wrote:



          > I think this notion of only ever doing things that have value to
          > the customer has been somewhat misunderstood, or misapplied. There
          > are most certainly things that need to be done that do not have an
          > immediate, direct, or even obvious value to a customer ... but
          > they still must be done. So what if user stories are not written
          > for them, and so what if they really don't fit the notion of user
          > stories (however defined). Being afraid to get it done because
          > Scrum doesn't seem to cover that situation just leads to
          > development paralysis.

          If one is to do Scrum well, one needs to understand Scrum well.
          Deviating from it every time something "seems" odd will not help a
          team to prosper. It troubles me that so much of your advice, Roy,
          deviates substantially from what Scrum and Agile teach, even when
          there are perfectly good Scrum / Agile answers.

          So here's so what.

          The fundamental point of Scrum is for the PO to prioritize work
          for the team, so that the best possible product can be delivered
          within the desired time and budget. The PO will get the best
          product if work is chosen that is of high "business value". (Low
          development cost is also a concern in choosing the work, but may
          will be substantially less important than value.)

          Clearly, the PO is capable of driving out more value when a larger
          percentage of the team's time comes under her control, since
          otherwise, she is directing a smaller flow of work, which will
          inherently reduce total value. Therefore it is undesirable to
          slice away a strip of the team's development time and make it
          unavailable for stories.

          However, so-called "technical stories" have a value proposition
          which is almost completely different from an ordinary story. A
          technical refactoring story may promise faster velocity sometime
          in the future, while an ordinary story is something that real
          users actually want. It is therefore very difficult, perhaps
          impossible, to compare an ordinary story and a technical story for
          value. Therefore it is undesirable to present technical stories at
          all, since they are so difficult to prioritize.

          This seems, if one doesn't think about it very hard, to be a
          dilemma. We can't take away the PO's time, we can't prioritize the
          technical stories. We can't have them at all. We are doomed.

          Like most dilemmas, this one is solved by using a third
          alternative. It turns out that [almost?] every bit of technical
          work, if it should be done at all, is done in support of one or
          more stories. IN PARTICULAR, the kind of work the OP is talking
          about, creation of tests and creation of clean code, is inherently
          part of creating stories.

          Why inherently, one might ask. Because for code to be DONE, it
          must work, and to know whether it works, we must test it. Because
          we are working incrementally, we will revisit much of the code
          many times and we might break it, so it is wise that the tests be
          recorded for reuse, i.e. automated.

          Further, since we are building the system incrementally, we are
          building the design incrementally. We do not have the time (nor,
          likely, the ability) to define the design correctly up front. We
          do not have the time to build it up front even if we did have the
          ability to define it. We must, perforce, evolve the design.
          Therefore, for each feature to be DONE, its design must be fit for
          the current purpose.

          Therefore, testing, and design improvement, are /inherently/ part
          of the work of doing stories.

          How does this help us with our dilemma? We have, through our human
          failings, fallen behind on the testing and design. What are we to
          do? We do not want to slice time away from the Product Owner. The
          Product Owner cannot correctly prioritize these strange technical
          stories. We are doomed.

          No. All we need to do is to bear down a bit on improving the code
          we work on, doing all it takes to bring it back up to the proper
          definition of DONE, namely "well-tested and well-factored". We
          make the code a bit better every time we pass through it.

          At first, maybe this takes a little longer per story. This is not
          a real concern, as no one but the team really knows if the team
          can do twelve this Sprint or only ten. The team commits to what
          they can do. They do it, and the world becomes a better place.

          Better yet, soon, surprisingly soon, it doesn't take longer after
          all. Since we are improving the code and tests in the areas we
          work in, velocity picks up quite quickly in the areas we work in.
          We wind up going faster than before, because we are going better.

          Our Product Owner retains her ability to prioritize the whole of
          our work, and our code improves. No confusion, no political
          discussion, no need to make promises about the future that we may
          not be able to keep.

          We don't need technical stories or tricky negotiations. We just do
          the job we were always supposed to do, namely make each story
          DONE, fully tested, with a good design in place.


          Ron Jeffries
          www.XProgramming.com
          Thousands of years ago, the first man discovered how to make fire.
          He was probably burned at the stake he had taught his brothers to
          light - Howard Roark (The Fountainhead, Ayn Rand)


        • Ron Jeffries
          Hello, Malcolm. On Wednesday, December 1, 2010, at 10:21:40 AM, ... Well, since clearly I would NOT encourage a lack of transparency, let s look a bit more
          Message 4 of 28 , Dec 1, 2010
            Hello, Malcolm. On Wednesday, December 1, 2010, at 10:21:40 AM,
            you wrote:

            > This issue in Scrum of how to keep up the technical maintenance has always
            > bothered me.

            > I was cruising along with your answer, singing praises to your
            > ability to make the unclear, clear. Until you got to the point
            > where you seem to encourage a lack of transparency.

            Well, since clearly I would NOT encourage a lack of transparency,
            let's look a bit more deeply at what's going on.

            >> At first, maybe this takes a little longer per story. This is not
            >> a real concern, as no one but the team really knows if the team
            >> can do twelve this Sprint or only ten. The team commits to what
            >> they can do. They do it, and the world becomes a better place.

            > This sounds like "lying to your product owner" (and the product
            > owners that I've worked with, do know that "the team can do
            > twelve") This goes against the value of transparency.

            Actually, the transparency violation has already occurred. The team
            told (and gave evidence) that they can do twelve, and screwed up the
            code base to do it. The team has produced software which everyone
            thought was DONE, but it was not DONE.

            The PO /thought/ they could do twelve. The PO was wrong. The team
            may have thought they could do twelve, and somehow accidentally
            wrote crappy code and too few tests in order to do it. More likely,
            the team knew quite well, long before this crisis arose, that they
            were doing crappy work.

            That is where the transparency violation occurs.

            The team demonstrably cannot do twelve stories up to the standard of
            DONE. They need to start doing as many as they can do.

            > It also takes away the PO's right to incur short term technical
            > debt in order to make a particular deadline.

            The PO does not have that right.

            The team agrees on the definition of DONE. If that definition does
            not include "well tested", then the software is not known to work.
            Software not known to work is not "potentially shippable". The team
            must, by the definition of Scrum, produce increments of work that
            are potentially shippable. Therefore, work must be well tested. This
            is not optional.

            The dev team decides how to do the work. The PO has absolutely no
            authority about how the work is to be done: the PO's authority is to
            prioritize the stories. The dev team decides how much can be done
            and how to do it.

            Furthermore, the PO does not have the ability to decide whether
            taking on technical debt will improve the chances of making a
            particular deadline, BECAUSE NO ONE HAS THAT ABILITY. There is no
            evidence that taking on technical debt for more than a few hours
            makes anything happen faster, since code containing technical debut
            is not known to work correctly and is known to work poorly.
            Maintaining poor code always takes longer than maintaining
            well-designed code.

            It is common to think, or hope, that writing crappy poorly tested
            code will somehow turn out to be better than writing good well
            tested code. This happens rarely, if at all, and we never have the
            ability to decide correctly whether this is our situation or not.

            > One of the weaknesses of Scrum is that we think, it's self evident
            > that product owners are responsible for maintaining the health
            > their product while they provide value to their customer.

            > I believe that this all comes down to Stewardship, and not making
            > it clear to the product owner that they need to be Stewards to
            > their product.

            Yes but I don't see your point regarding how the product is built.
            That is simply not part of the PO's job in Scrum. I can think of few
            situations where it would be, since in most situations the PO is not
            technically qualified.

            > I've got 3 questions, the answers to which I believe are currently "No, No
            > and No"

            > 1) Is there a Product Owner body where Product Owners discuss with
            > other product owners how to be a better product owner?

            > 2) Is the concept of Stewardship a part of the PO training?

            > 3) Has the PO body taking on a commitment to maintaining and improving the
            > health of their products?

            As far as I know the answers are no. And, the PO job is already
            incredibly difficult. Making them responsible as well for the
            "health" of the product -- assuming that by that you mean the
            technical quality -- will add the burden of being expert in software
            development to an already very difficult job.

            If done, this would also be a different thing from Scrum. Scrum's
            design has a strict line between what, the domain of the Product
            Owner, and how, the domain of the Development Team. Perhaps a
            framework could be defined that would be better than Scrum that did
            not have this line, but it would not be Scrum, it would be some new
            thing.

            > Until the answer to the above 3 questions is Yes, Yes, and Yes, this is
            > going to be an issue with Scrum.
            > Until we have some kind of "Product Owner Oath of Stewardship," this is
            > going to be an issue with Scrum.
            > Until stewardship is self evident to people other than Scrum Masters, this
            > is going to be an issue with Scrum.

            Actually, I disagree that this is the key issue around technical
            quality.

            Technical quality should be in the hands of the people most
            qualified to provide it, just as the choice of what features to do
            and which to leave out is in the hands of the people most qualified
            to do it.

            There are many things one could object to about Scrum. The existence
            of the ScrumMaster role, for example. But one thing that Scrum has
            pretty darn right is the notion of creating teams with the business
            people deciding what, and technical people deciding how.

            Ron Jeffries
            www.XProgramming.com
            Yesterday's code should be as good as we could make it yesterday.
            The fact that we know more today, and are more capable today,
            is good news about today, not bad news about yesterday.
          • Alan Dayley
            Interesting point of view, Malcolm. You may be pointing out a lack in emphasis on the Product Owner role in many training and use situations. I don t think
            Message 5 of 28 , Dec 1, 2010
              Interesting point of view, Malcolm.  You may be pointing out a lack in emphasis on the Product Owner role in many training and use situations.  I don't think it is a fault of the Scrum framework, however.

              Phrases seen in Scrum writings with regard to the Product Owner role:  "...owns the ROI of the product..."  "..represents the customer..."  "...owns the Product Backlog..." and the odious "...single wringable neck..."  And even the title of the role means *owner* of the product.

              What part of these terms does not mean stewardship?  You are pointing out that Product Owners are not getting the message, *despite* what the Scrum training and literature states.  Maybe an oath would help but I don't think the message of product stewardship is lacking in Scrum.

              I had some other ideas to write but Ron was faster with his elaborations, which I don't think I can say any better.

              Nice discussions!

              Alan

              On Wed, Dec 1, 2010 at 8:21 AM, Malcolm Anderson <malcolm.b.anderson@...> wrote:
               

              Ron

              This issue in Scrum of how to keep up the technical maintenance has always bothered me.

              I was cruising along with your answer, singing praises to your ability to make the unclear, clear. 
              Until you got to the point where you seem to encourage a lack of transparency.



              >At first, maybe this takes a little longer per story. This is not a real concern, as no one but the team really knows if the team
              >can do twelve this Sprint or only ten. The team commits to what they can do. They do it, and the world becomes a better place.

              This sounds like "lying to your product owner" (and the product owners that I've worked with, do know that "the team can do twelve")
              This goes against the value of transparency. 
              It also takes away the PO's right to incur short term technical debt in order to make a particular deadline.


              One of the weaknesses of Scrum is that we think, it's self evident that product owners are responsible for maintaining the health their product while they provide value to their customer.

              I believe that this all comes down to Stewardship, and not making it clear to the product owner that they need to be Stewards to their product.

              I've got 3 questions, the answers to which I believe are currently "No, No and No"

              1) Is there a Product Owner body where Product Owners discuss with other product owners how to be a better product owner?

              2) Is the concept of Stewardship a part of the PO training?

              3) Has the PO body taking on a commitment to maintaining and improving the health of their products?


              Until the answer to the above 3 questions is Yes, Yes, and Yes, this is going to be an issue with Scrum.
              Until we have some kind of "Product Owner Oath of Stewardship," this is going to be an issue with Scrum.
              Until stewardship is self evident to people other than Scrum Masters, this is going to be an issue with Scrum.


              Thanks

              Malcolm Anderson
              Scrum Coach & Agile Engineer





              On Wed, Dec 1, 2010 at 6:16 AM, Ron Jeffries <ronjeffries@...> wrote:
               

              Hello, Roy. On Wednesday, December 1, 2010, at 3:29:27 AM, you
              wrote:



              > I think this notion of only ever doing things that have value to
              > the customer has been somewhat misunderstood, or misapplied. There
              > are most certainly things that need to be done that do not have an
              > immediate, direct, or even obvious value to a customer ... but
              > they still must be done. So what if user stories are not written
              > for them, and so what if they really don't fit the notion of user
              > stories (however defined). Being afraid to get it done because
              > Scrum doesn't seem to cover that situation just leads to
              > development paralysis.

              If one is to do Scrum well, one needs to understand Scrum well.
              Deviating from it every time something "seems" odd will not help a
              team to prosper. It troubles me that so much of your advice, Roy,
              deviates substantially from what Scrum and Agile teach, even when
              there are perfectly good Scrum / Agile answers.

              So here's so what.

              The fundamental point of Scrum is for the PO to prioritize work
              for the team, so that the best possible product can be delivered
              within the desired time and budget. The PO will get the best
              product if work is chosen that is of high "business value". (Low
              development cost is also a concern in choosing the work, but may
              will be substantially less important than value.)

              Clearly, the PO is capable of driving out more value when a larger
              percentage of the team's time comes under her control, since
              otherwise, she is directing a smaller flow of work, which will
              inherently reduce total value. Therefore it is undesirable to
              slice away a strip of the team's development time and make it
              unavailable for stories.

              However, so-called "technical stories" have a value proposition
              which is almost completely different from an ordinary story. A
              technical refactoring story may promise faster velocity sometime
              in the future, while an ordinary story is something that real
              users actually want. It is therefore very difficult, perhaps
              impossible, to compare an ordinary story and a technical story for
              value. Therefore it is undesirable to present technical stories at
              all, since they are so difficult to prioritize.

              This seems, if one doesn't think about it very hard, to be a
              dilemma. We can't take away the PO's time, we can't prioritize the
              technical stories. We can't have them at all. We are doomed.

              Like most dilemmas, this one is solved by using a third
              alternative. It turns out that [almost?] every bit of technical
              work, if it should be done at all, is done in support of one or
              more stories. IN PARTICULAR, the kind of work the OP is talking
              about, creation of tests and creation of clean code, is inherently
              part of creating stories.

              Why inherently, one might ask. Because for code to be DONE, it
              must work, and to know whether it works, we must test it. Because
              we are working incrementally, we will revisit much of the code
              many times and we might break it, so it is wise that the tests be
              recorded for reuse, i.e. automated.

              Further, since we are building the system incrementally, we are
              building the design incrementally. We do not have the time (nor,
              likely, the ability) to define the design correctly up front. We
              do not have the time to build it up front even if we did have the
              ability to define it. We must, perforce, evolve the design.
              Therefore, for each feature to be DONE, its design must be fit for
              the current purpose.

              Therefore, testing, and design improvement, are /inherently/ part
              of the work of doing stories.

              How does this help us with our dilemma? We have, through our human
              failings, fallen behind on the testing and design. What are we to
              do? We do not want to slice time away from the Product Owner. The
              Product Owner cannot correctly prioritize these strange technical
              stories. We are doomed.

              No. All we need to do is to bear down a bit on improving the code
              we work on, doing all it takes to bring it back up to the proper
              definition of DONE, namely "well-tested and well-factored". We
              make the code a bit better every time we pass through it.

              At first, maybe this takes a little longer per story. This is not
              a real concern, as no one but the team really knows if the team
              can do twelve this Sprint or only ten. The team commits to what
              they can do. They do it, and the world becomes a better place.

              Better yet, soon, surprisingly soon, it doesn't take longer after
              all. Since we are improving the code and tests in the areas we
              work in, velocity picks up quite quickly in the areas we work in.
              We wind up going faster than before, because we are going better.

              Our Product Owner retains her ability to prioritize the whole of
              our work, and our code improves. No confusion, no political
              discussion, no need to make promises about the future that we may
              not be able to keep.

              We don't need technical stories or tricky negotiations. We just do
              the job we were always supposed to do, namely make each story
              DONE, fully tested, with a good design in place.


              Ron Jeffries
              www.XProgramming.com
              Thousands of years ago, the first man discovered how to make fire.
              He was probably burned at the stake he had taught his brothers to
              light - Howard Roark (The Fountainhead, Ayn Rand)



            • Malcolm Anderson
              Hi Ron I appreciate your focused attention on this. This got long, and is looking like it s to be a conversation held over beer. Responses inline Malcolm ...
              Message 6 of 28 , Dec 1, 2010
                Hi Ron

                I appreciate your focused attention on this. This got long, and is
                looking like it's to be a conversation held over beer.

                Responses inline

                Malcolm



                On Wed, Dec 1, 2010 at 10:40 AM, Ron Jeffries <ronjeffries@...> wrote:
                > Hello, Malcolm.  On Wednesday, December 1, 2010, at 10:21:40 AM,
                > you wrote:
                >
                >> This issue in Scrum of how to keep up the technical maintenance has always
                >> bothered me.
                >
                >> I was cruising along with your answer, singing praises to your
                >> ability to make the unclear, clear. Until you got to the point
                >> where you seem to encourage a lack of transparency.
                >
                > Well, since clearly I would NOT encourage a lack of transparency,
                > let's look a bit more deeply at what's going on.

                I didn't figure that you would be. :)

                >>> At first, maybe this takes a little longer per story. This is not
                >>> a real concern, as no one but the team really knows if the team
                >>> can do twelve this Sprint or only ten. The team commits to what
                >>> they can do. They do it, and the world becomes a better place.
                >
                >> This sounds like "lying to your product owner" (and the product
                >> owners that I've worked with, do know that "the team can do
                >> twelve") This goes against the value of transparency.
                >
                > Actually, the transparency violation has already occurred. The team
                > told (and gave evidence) that they can do twelve, and screwed up the
                > code base to do it. The team has produced software which everyone
                > thought was DONE, but it was not DONE.

                Here is an issue that seems to be at the base line. I'm working with
                experienced teams that know nothing of the agile engineering
                practices. They do NOT have an agreed upon definition of DONE. Here,
                on this list, (in what I say is the "Scrum Community"), we expect that
                Test First TDD will be part of the done criteria. We expect that the
                team will be working with some kind of continuous integration. We
                might even expect that they are doing Paired Programming, or at least
                code reviews. We expect that people are using design patterns and
                doing object oriented programming.

                I can tell you that as Scrum becomes more popular, we as coaches and
                trainers are going to start running into more and more large
                established companies for whom "Agile Engineering" is more of a
                punchline than it is, a self evident body of proven practices.

                So here is a foundational disagreement. I do not believe that there
                is any consensus out there about "what constitutes DONE." It sounds
                to me, like you do believe there is some consensus around the matter.



                >
                >> It also takes away the PO's right to incur short term technical
                >> debt in order to make a particular deadline.
                >
                > The PO does not have that right.

                I disagree. There are a lot of reasons for taking on short term
                technical debt. I use the analogy of the credit card with a 36%
                interest rate. You do not want to use that card. But there are times
                when it makes great business sense to use that card in a short term
                situation.



                > The team agrees on the definition of DONE. If that definition does
                > not include "well tested", then the software is not known to work.
                > Software not known to work is not "potentially shippable". The team
                > must, by the definition of Scrum, produce increments of work that
                > are potentially shippable. Therefore, work must be well tested. This
                > is not optional.

                Again, I agree with you. Your definition of DONE is self evident to
                me. I disagree with you that it is self evident to the majority of
                the developers out in the world. I disagree with you that it is self
                evident to more than 3% of project managers. And even if it is self
                evident to them, they will tell you, "there's no way that that will
                ever work here; too many cowboys, and our code is too fragile" (which
                just seems self inflicted)

                > The dev team decides how to do the work. The PO has absolutely no
                > authority about how the work is to be done: the PO's authority is to
                > prioritize the stories. The dev team decides how much can be done
                > and how to do it.

                Agreed

                > Furthermore, the PO does not have the ability to decide whether
                > taking on technical debt will improve the chances of making a
                > particular deadline, BECAUSE NO ONE HAS THAT ABILITY. There is no
                > evidence that taking on technical debt for more than a few hours
                > makes anything happen faster, since code containing technical debut
                > is not known to work correctly and is known to work poorly.
                > Maintaining poor code always takes longer than maintaining
                > well-designed code.

                Agreed in spirit. Still think there are times when it makes sense to
                "be stupid on purpose."

                > It is common to think, or hope, that writing crappy poorly tested
                > code will somehow turn out to be better than writing good well
                > tested code. This happens rarely, if at all, and we never have the
                > ability to decide correctly whether this is our situation or not.

                Agreed. Self evident even. To me and you.

                >> One of the weaknesses of Scrum is that we think, it's self evident
                >> that product owners are responsible for maintaining the health
                >> their product while they provide value to their customer.
                >
                >> I believe that this all comes down to Stewardship, and not making
                >> it clear to the product owner that they need to be Stewards to
                >> their product.
                >
                > Yes but I don't see your point regarding how the product is built.
                > That is simply not part of the PO's job in Scrum. I can think of few
                > situations where it would be, since in most situations the PO is not
                > technically qualified.

                I disagree strongly with you here. The PO doesn't have to be
                technical to take responsibility for the health of his or her project.

                The PO does need to believe that the agile engineering practices can
                make a dramatic difference in the quality and responsiveness of their
                code base. Again, its self evident to me, and obviously to you.

                A non technical PO that takes on the responsibility of maintaining and
                improving the health of their products code will end up with better
                code than a technical PO that doesn't believe that there is value in
                improving the health of your code.

                Case in point, recently I worked with a very bright, very technical
                product owner who believed that code quality could only come from more
                complete up front analysis before coding ever starts. ("recently" in
                this sentence means 2010)


                >> I've got 3 questions, the answers to which I believe are currently "No, No
                >> and No"
                >
                >> 1) Is there a Product Owner body where Product Owners discuss with
                >> other product owners how to be a better product owner?
                >
                >> 2) Is the concept of Stewardship a part of the PO training?
                >
                >> 3) Has the PO body taking on a commitment to maintaining and improving the
                >> health of their products?
                >
                > As far as I know the answers are no. And, the PO job is already
                > incredibly difficult. Making them responsible as well for the
                > "health" of the product -- assuming that by that you mean the
                > technical quality -- will add the burden of being expert in software
                > development to an already very difficult job.

                I hoped that you were going to surprise me and tell me about the
                thriving ScrumProductOwnersRock yahoo group

                Again, we disagree here. I believe that non-technical POs can do a
                great job in promoting improved health of their code base.



                >
                > If done, this would also be a different thing from Scrum. Scrum's
                > design has a strict line between what, the domain of the Product
                > Owner, and how, the domain of the Development Team. Perhaps a
                > framework could be defined that would be better than Scrum that did
                > not have this line, but it would not be Scrum, it would be some new
                > thing.

                I disagree with you. I do not think that having the Product Owner
                have some care about the codes health contradicts anything in Scrum.


                >> Until the answer to the above 3 questions is Yes, Yes, and Yes, this is
                >> going to be an issue with Scrum.
                >> Until we have some kind of "Product Owner Oath of Stewardship," this is
                >> going to be an issue with Scrum.
                >> Until stewardship is self evident to people other than Scrum Masters, this
                >> is going to be an issue with Scrum.
                >
                > Actually, I disagree that this is the key issue around technical
                > quality.
                >
                > Technical quality should be in the hands of the people most
                > qualified to provide it, just as the choice of what features to do
                > and which to leave out is in the hands of the people most qualified
                > to do it.

                I'm an agile engineer, I expect that teams will see value by taking on
                some simple disciplines. I've dealt with Product Owners who don't see
                it that way. They see a cost today, that they do not believe will
                bring them rewards tomorrow. The data on Paired Programming is
                overwhelming, I don't believe it's possible to do research on it and
                come away not wanting to try it. But it's counter intuitive, "You
                want me to put two people on a task that one should be able to handle?
                I can't afford that kind of luxury." Some times that quote comes
                from technical people.


                >
                > There are many things one could object to about Scrum. The existence
                > of the ScrumMaster role, for example. But one thing that Scrum has
                > pretty darn right is the notion of creating teams with the business
                > people deciding what, and technical people deciding how.

                I guess it comes down to this Ron. I believe that a commitment to
                product stewardship is a "what" and falls on the PO to support their
                team in taking the time to decide the how of improving their products
                code and design.
              • Malcolm Anderson
                Thanks Alan I agree, if you read between the lines, stewardship is already demanded. But if PO s are not getting the message then we need inspect and adapt
                Message 7 of 28 , Dec 1, 2010
                  Thanks Alan

                  I agree, if you read between the lines, stewardship is already demanded.  But if PO's are not getting the message then we need inspect and adapt *how* we are sending that message.

                  This is made more difficult because more and more POs are being drafted from the PM ranks. 
                  I'm still seeing PMs drafted as Scrum Masters (because everyone knows that "Scrum Master" is Agile for "Project Manager".)  Thankfully for them, no one knows what a Scrum Master is, or does, so they HAVE to do research on it.

                  (I just saw a job requirement yesterday that was looking for a PMP certified project manager "with strong agile and scrum experience."  There was no requirement for Certified Scrum Master, and certainly no requirement for Certified Product Owner.  This horrified me, but it's a sign of our times)

                  As Scrum becomes more popular, we will have to re-educate more and more people. 
                  The word "stewardship" clearly emphasize something that the scrum body takes for granted.

                  I was kind of joking when I wrote the phrase "Product Owner Oath of Stewardship" but maybe it's not a bad idea. 

                  Malcolm


                  On Wed, Dec 1, 2010 at 10:48 AM, Alan Dayley <alandd@...> wrote:
                   

                  Interesting point of view, Malcolm.  You may be pointing out a lack in emphasis on the Product Owner role in many training and use situations.  I don't think it is a fault of the Scrum framework, however.


                  Phrases seen in Scrum writings with regard to the Product Owner role:  "...owns the ROI of the product..."  "..represents the customer..."  "...owns the Product Backlog..." and the odious "...single wringable neck..."  And even the title of the role means *owner* of the product.

                  What part of these terms does not mean stewardship?  You are pointing out that Product Owners are not getting the message, *despite* what the Scrum training and literature states.  Maybe an oath would help but I don't think the message of product stewardship is lacking in Scrum.

                  I had some other ideas to write but Ron was faster with his elaborations, which I don't think I can say any better.

                  Nice discussions!

                  Alan



                • Malcolm Anderson
                  Ron, Alan I probably should not blog at midnight. http://geekswithblogs.net/geekusconlivus/archive/2010/12/01/scrum-product-owners-oath-of-stewardship.aspx
                  Message 8 of 28 , Dec 1, 2010
                    Ron, Alan

                    I probably should not blog at midnight.

                    http://geekswithblogs.net/geekusconlivus/archive/2010/12/01/scrum-product-owners-oath-of-stewardship.aspx

                    Ron, you'll get a track back too, I promise.

                    Malcolm



                    On Wed, Dec 1, 2010 at 10:48 AM, Alan Dayley <alandd@...> wrote:
                     

                    Interesting point of view, Malcolm.  You may be pointing out a lack in emphasis on the Product Owner role in many training and use situations.  I don't think it is a fault of the Scrum framework, however.


                    Phrases seen in Scrum writings with regard to the Product Owner role:  "...owns the ROI of the product..."  "..represents the customer..."  "...owns the Product Backlog..." and the odious "...single wringable neck..."  And even the title of the role means *owner* of the product.

                    What part of these terms does not mean stewardship?  You are pointing out that Product Owners are not getting the message, *despite* what the Scrum training and literature states.  Maybe an oath would help but I don't think the message of product stewardship is lacking in Scrum.

                    I had some other ideas to write but Ron was faster with his elaborations, which I don't think I can say any better.

                    Nice discussions!

                    Alan

                    On Wed, Dec 1, 2010 at 8:21 AM, Malcolm Anderson <malcolm.b.anderson@...> wrote:
                     

                    Ron

                    This issue in Scrum of how to keep up the technical maintenance has always bothered me.

                    I was cruising along with your answer, singing praises to your ability to make the unclear, clear. 
                    Until you got to the point where you seem to encourage a lack of transparency.



                    >At first, maybe this takes a little longer per story. This is not a real concern, as no one but the team really knows if the team
                    >can do twelve this Sprint or only ten. The team commits to what they can do. They do it, and the world becomes a better place.

                    This sounds like "lying to your product owner" (and the product owners that I've worked with, do know that "the team can do twelve")
                    This goes against the value of transparency. 
                    It also takes away the PO's right to incur short term technical debt in order to make a particular deadline.


                    One of the weaknesses of Scrum is that we think, it's self evident that product owners are responsible for maintaining the health their product while they provide value to their customer.

                    I believe that this all comes down to Stewardship, and not making it clear to the product owner that they need to be Stewards to their product.

                    I've got 3 questions, the answers to which I believe are currently "No, No and No"

                    1) Is there a Product Owner body where Product Owners discuss with other product owners how to be a better product owner?

                    2) Is the concept of Stewardship a part of the PO training?

                    3) Has the PO body taking on a commitment to maintaining and improving the health of their products?


                    Until the answer to the above 3 questions is Yes, Yes, and Yes, this is going to be an issue with Scrum.
                    Until we have some kind of "Product Owner Oath of Stewardship," this is going to be an issue with Scrum.
                    Until stewardship is self evident to people other than Scrum Masters, this is going to be an issue with Scrum.


                    Thanks

                    Malcolm Anderson
                    Scrum Coach & Agile Engineer





                    On Wed, Dec 1, 2010 at 6:16 AM, Ron Jeffries <ronjeffries@...> wrote:
                     

                    Hello, Roy. On Wednesday, December 1, 2010, at 3:29:27 AM, you
                    wrote:



                    > I think this notion of only ever doing things that have value to
                    > the customer has been somewhat misunderstood, or misapplied. There
                    > are most certainly things that need to be done that do not have an
                    > immediate, direct, or even obvious value to a customer ... but
                    > they still must be done. So what if user stories are not written
                    > for them, and so what if they really don't fit the notion of user
                    > stories (however defined). Being afraid to get it done because
                    > Scrum doesn't seem to cover that situation just leads to
                    > development paralysis.

                    If one is to do Scrum well, one needs to understand Scrum well.
                    Deviating from it every time something "seems" odd will not help a
                    team to prosper. It troubles me that so much of your advice, Roy,
                    deviates substantially from what Scrum and Agile teach, even when
                    there are perfectly good Scrum / Agile answers.

                    So here's so what.

                    The fundamental point of Scrum is for the PO to prioritize work
                    for the team, so that the best possible product can be delivered
                    within the desired time and budget. The PO will get the best
                    product if work is chosen that is of high "business value". (Low
                    development cost is also a concern in choosing the work, but may
                    will be substantially less important than value.)

                    Clearly, the PO is capable of driving out more value when a larger
                    percentage of the team's time comes under her control, since
                    otherwise, she is directing a smaller flow of work, which will
                    inherently reduce total value. Therefore it is undesirable to
                    slice away a strip of the team's development time and make it
                    unavailable for stories.

                    However, so-called "technical stories" have a value proposition
                    which is almost completely different from an ordinary story. A
                    technical refactoring story may promise faster velocity sometime
                    in the future, while an ordinary story is something that real
                    users actually want. It is therefore very difficult, perhaps
                    impossible, to compare an ordinary story and a technical story for
                    value. Therefore it is undesirable to present technical stories at
                    all, since they are so difficult to prioritize.

                    This seems, if one doesn't think about it very hard, to be a
                    dilemma. We can't take away the PO's time, we can't prioritize the
                    technical stories. We can't have them at all. We are doomed.

                    Like most dilemmas, this one is solved by using a third
                    alternative. It turns out that [almost?] every bit of technical
                    work, if it should be done at all, is done in support of one or
                    more stories. IN PARTICULAR, the kind of work the OP is talking
                    about, creation of tests and creation of clean code, is inherently
                    part of creating stories.

                    Why inherently, one might ask. Because for code to be DONE, it
                    must work, and to know whether it works, we must test it. Because
                    we are working incrementally, we will revisit much of the code
                    many times and we might break it, so it is wise that the tests be
                    recorded for reuse, i.e. automated.

                    Further, since we are building the system incrementally, we are
                    building the design incrementally. We do not have the time (nor,
                    likely, the ability) to define the design correctly up front. We
                    do not have the time to build it up front even if we did have the
                    ability to define it. We must, perforce, evolve the design.
                    Therefore, for each feature to be DONE, its design must be fit for
                    the current purpose.

                    Therefore, testing, and design improvement, are /inherently/ part
                    of the work of doing stories.

                    How does this help us with our dilemma? We have, through our human
                    failings, fallen behind on the testing and design. What are we to
                    do? We do not want to slice time away from the Product Owner. The
                    Product Owner cannot correctly prioritize these strange technical
                    stories. We are doomed.

                    No. All we need to do is to bear down a bit on improving the code
                    we work on, doing all it takes to bring it back up to the proper
                    definition of DONE, namely "well-tested and well-factored". We
                    make the code a bit better every time we pass through it.

                    At first, maybe this takes a little longer per story. This is not
                    a real concern, as no one but the team really knows if the team
                    can do twelve this Sprint or only ten. The team commits to what
                    they can do. They do it, and the world becomes a better place.

                    Better yet, soon, surprisingly soon, it doesn't take longer after
                    all. Since we are improving the code and tests in the areas we
                    work in, velocity picks up quite quickly in the areas we work in.
                    We wind up going faster than before, because we are going better.

                    Our Product Owner retains her ability to prioritize the whole of
                    our work, and our code improves. No confusion, no political
                    discussion, no need to make promises about the future that we may
                    not be able to keep.

                    We don't need technical stories or tricky negotiations. We just do
                    the job we were always supposed to do, namely make each story
                    DONE, fully tested, with a good design in place.


                    Ron Jeffries
                    www.XProgramming.com
                    Thousands of years ago, the first man discovered how to make fire.
                    He was probably burned at the stake he had taught his brothers to
                    light - Howard Roark (The Fountainhead, Ayn Rand)




                  • peterskeide
                    I think you have some good points here. Like you, I feel that sometimes baking in some technical work in a User Story is like hiding information, especially
                    Message 9 of 28 , Dec 2, 2010
                      I think you have some good points here. Like you, I feel that sometimes "baking in" some technical work in a User Story is like hiding information, especially if the technical issues and the User Story are unrelated.

                      It is important to remember that developers are stakeholders too. That's why I don't see a problem with the product backlog sometimes reflecting the needs of the developers.

                      In a perfect world everybody get to work on exciting new products, know exactly what to do and how to do it well. Unfortunately, that just isn't so for most of us. Sometimes we must work on legacy systems where there never was a clear definition of done. Sometimes we must help organizations/projects that are transitioning to agile and just learning practices that we as a community take for granted. In such situations it can be very important to highlight issues related to technical debt or developer supporting infrastructure. I think the Product Backlog is one way to make such issues visible.

                      Also, don't forget that Scrum does not mandate User Stories. For some, technical requirements may be just the thing. Others prefer Use Cases, or perhaps measurable stakeholder goals (as promoted by Gilb). Hopefully, everyone takes their own context into consideration and then *choose the format best suited to their current needs*.

                      --- In scrumdevelopment@yahoogroups.com, Malcolm Anderson <malcolm.b.anderson@...> wrote:
                      >
                      > Ron
                      >
                      > This issue in Scrum of how to keep up the technical maintenance has always
                      > bothered me.
                      >
                      > I was cruising along with your answer, singing praises to your ability to
                      > make the unclear, clear.
                      > Until you got to the point where you seem to encourage a lack of
                      > transparency.
                      >
                      > >At first, maybe this takes a little longer per story. This is not a real
                      > concern, as no one but the team really knows if the team
                      > >can do twelve this Sprint or only ten. The team commits to what they can
                      > do. They do it, and the world becomes a better place.
                      >
                      > This sounds like "lying to your product owner" (and the product owners that
                      > I've worked with, do know that "the team can do twelve")
                      > This goes against the value of transparency.
                      > It also takes away the PO's right to incur short term technical debt in
                      > order to make a particular deadline.
                      >
                      >
                      > One of the weaknesses of Scrum is that we think, it's self evident that
                      > product owners are responsible for maintaining the health their product
                      > while they provide value to their customer.
                      >
                      > I believe that this all comes down to Stewardship, and not making it clear
                      > to the product owner that they need to be Stewards to their product.
                      >
                      > I've got 3 questions, the answers to which I believe are currently "No, No
                      > and No"
                      >
                      > 1) Is there a Product Owner body where Product Owners discuss with other
                      > product owners how to be a better product owner?
                      >
                      > 2) Is the concept of Stewardship a part of the PO training?
                      >
                      > 3) Has the PO body taking on a commitment to maintaining and improving the
                      > health of their products?
                      >
                      >
                      > Until the answer to the above 3 questions is Yes, Yes, and Yes, this is
                      > going to be an issue with Scrum.
                      > Until we have some kind of "Product Owner Oath of Stewardship," this is
                      > going to be an issue with Scrum.
                      > Until stewardship is self evident to people other than Scrum Masters, this
                      > is going to be an issue with Scrum.
                      >
                      >
                      > Thanks
                      >
                      > Malcolm Anderson
                      > Scrum Coach & Agile Engineer
                      >
                      >
                      >
                      >
                      > On Wed, Dec 1, 2010 at 6:16 AM, Ron Jeffries <ronjeffries@...> wrote:
                      >
                      > >
                      > >
                      > > Hello, Roy. On Wednesday, December 1, 2010, at 3:29:27 AM, you
                      > > wrote:
                      > >
                      > >
                      > > > I think this notion of only ever doing things that have value to
                      > > > the customer has been somewhat misunderstood, or misapplied. There
                      > > > are most certainly things that need to be done that do not have an
                      > > > immediate, direct, or even obvious value to a customer ... but
                      > > > they still must be done. So what if user stories are not written
                      > > > for them, and so what if they really don't fit the notion of user
                      > > > stories (however defined). Being afraid to get it done because
                      > > > Scrum doesn't seem to cover that situation just leads to
                      > > > development paralysis.
                      > >
                      > > If one is to do Scrum well, one needs to understand Scrum well.
                      > > Deviating from it every time something "seems" odd will not help a
                      > > team to prosper. It troubles me that so much of your advice, Roy,
                      > > deviates substantially from what Scrum and Agile teach, even when
                      > > there are perfectly good Scrum / Agile answers.
                      > >
                      > > So here's so what.
                      > >
                      > > The fundamental point of Scrum is for the PO to prioritize work
                      > > for the team, so that the best possible product can be delivered
                      > > within the desired time and budget. The PO will get the best
                      > > product if work is chosen that is of high "business value". (Low
                      > > development cost is also a concern in choosing the work, but may
                      > > will be substantially less important than value.)
                      > >
                      > > Clearly, the PO is capable of driving out more value when a larger
                      > > percentage of the team's time comes under her control, since
                      > > otherwise, she is directing a smaller flow of work, which will
                      > > inherently reduce total value. Therefore it is undesirable to
                      > > slice away a strip of the team's development time and make it
                      > > unavailable for stories.
                      > >
                      > > However, so-called "technical stories" have a value proposition
                      > > which is almost completely different from an ordinary story. A
                      > > technical refactoring story may promise faster velocity sometime
                      > > in the future, while an ordinary story is something that real
                      > > users actually want. It is therefore very difficult, perhaps
                      > > impossible, to compare an ordinary story and a technical story for
                      > > value. Therefore it is undesirable to present technical stories at
                      > > all, since they are so difficult to prioritize.
                      > >
                      > > This seems, if one doesn't think about it very hard, to be a
                      > > dilemma. We can't take away the PO's time, we can't prioritize the
                      > > technical stories. We can't have them at all. We are doomed.
                      > >
                      > > Like most dilemmas, this one is solved by using a third
                      > > alternative. It turns out that [almost?] every bit of technical
                      > > work, if it should be done at all, is done in support of one or
                      > > more stories. IN PARTICULAR, the kind of work the OP is talking
                      > > about, creation of tests and creation of clean code, is inherently
                      > > part of creating stories.
                      > >
                      > > Why inherently, one might ask. Because for code to be DONE, it
                      > > must work, and to know whether it works, we must test it. Because
                      > > we are working incrementally, we will revisit much of the code
                      > > many times and we might break it, so it is wise that the tests be
                      > > recorded for reuse, i.e. automated.
                      > >
                      > > Further, since we are building the system incrementally, we are
                      > > building the design incrementally. We do not have the time (nor,
                      > > likely, the ability) to define the design correctly up front. We
                      > > do not have the time to build it up front even if we did have the
                      > > ability to define it. We must, perforce, evolve the design.
                      > > Therefore, for each feature to be DONE, its design must be fit for
                      > > the current purpose.
                      > >
                      > > Therefore, testing, and design improvement, are /inherently/ part
                      > > of the work of doing stories.
                      > >
                      > > How does this help us with our dilemma? We have, through our human
                      > > failings, fallen behind on the testing and design. What are we to
                      > > do? We do not want to slice time away from the Product Owner. The
                      > > Product Owner cannot correctly prioritize these strange technical
                      > > stories. We are doomed.
                      > >
                      > > No. All we need to do is to bear down a bit on improving the code
                      > > we work on, doing all it takes to bring it back up to the proper
                      > > definition of DONE, namely "well-tested and well-factored". We
                      > > make the code a bit better every time we pass through it.
                      > >
                      > > At first, maybe this takes a little longer per story. This is not
                      > > a real concern, as no one but the team really knows if the team
                      > > can do twelve this Sprint or only ten. The team commits to what
                      > > they can do. They do it, and the world becomes a better place.
                      > >
                      > > Better yet, soon, surprisingly soon, it doesn't take longer after
                      > > all. Since we are improving the code and tests in the areas we
                      > > work in, velocity picks up quite quickly in the areas we work in.
                      > > We wind up going faster than before, because we are going better.
                      > >
                      > > Our Product Owner retains her ability to prioritize the whole of
                      > > our work, and our code improves. No confusion, no political
                      > > discussion, no need to make promises about the future that we may
                      > > not be able to keep.
                      > >
                      > > We don't need technical stories or tricky negotiations. We just do
                      > > the job we were always supposed to do, namely make each story
                      > > DONE, fully tested, with a good design in place.
                      > >
                      > >
                      > > Ron Jeffries
                      > > www.XProgramming.com
                      > > Thousands of years ago, the first man discovered how to make fire.
                      > > He was probably burned at the stake he had taught his brothers to
                      > > light - Howard Roark (The Fountainhead, Ayn Rand)
                      > >
                      > >
                      > >
                      >
                    • Ron Jeffries
                      Hello, peterskeide. On Thursday, December 2, 2010, at 6:25:41 AM, ... I would not recommend that. I don t know anyone who would. Ron Jeffries
                      Message 10 of 28 , Dec 2, 2010
                        Hello, peterskeide. On Thursday, December 2, 2010, at 6:25:41 AM,
                        you wrote:

                        > I think you have some good points here. Like you, I feel that
                        > sometimes "baking in" some technical work in a User Story is like
                        > hiding information, especially if the technical issues and the
                        > User Story are unrelated.

                        I would not recommend that. I don't know anyone who would.

                        Ron Jeffries
                        www.XProgramming.com
                        If you don't push something beyond its boundary of usefulness
                        how do you find where that boundary is? -- Martin Fowler
                      • Alan Dayley
                        Nice discussion of the thoughts, Malcolm. I m flattered. An Oath of Stewardship can be a useful framework for discussion of interaction between the Product
                        Message 11 of 28 , Dec 2, 2010
                          Nice discussion of the thoughts, Malcolm.  I'm flattered.

                          An Oath of Stewardship can be a useful framework for discussion of interaction between the Product Owner and the developers on the team.  And I may use it as such.  I'd not ever support such a thing be promulgated as mandatory.

                          Alan

                          On Wed, Dec 1, 2010 at 11:00 PM, Malcolm Anderson <malcolm.b.anderson@...> wrote:
                           

                          Ron, Alan

                          I probably should not blog at midnight.

                          http://geekswithblogs.net/geekusconlivus/archive/2010/12/01/scrum-product-owners-oath-of-stewardship.aspx

                          Ron, you'll get a track back too, I promise.

                          Malcolm



                          On Wed, Dec 1, 2010 at 10:48 AM, Alan Dayley <alandd@...> wrote:
                           

                          Interesting point of view, Malcolm.  You may be pointing out a lack in emphasis on the Product Owner role in many training and use situations.  I don't think it is a fault of the Scrum framework, however.


                          Phrases seen in Scrum writings with regard to the Product Owner role:  "...owns the ROI of the product..."  "..represents the customer..."  "...owns the Product Backlog..." and the odious "...single wringable neck..."  And even the title of the role means *owner* of the product.

                          What part of these terms does not mean stewardship?  You are pointing out that Product Owners are not getting the message, *despite* what the Scrum training and literature states.  Maybe an oath would help but I don't think the message of product stewardship is lacking in Scrum.

                          I had some other ideas to write but Ron was faster with his elaborations, which I don't think I can say any better.

                          Nice discussions!

                          Alan

                          On Wed, Dec 1, 2010 at 8:21 AM, Malcolm Anderson <malcolm.b.anderson@...> wrote:
                           

                          Ron

                          This issue in Scrum of how to keep up the technical maintenance has always bothered me.

                          I was cruising along with your answer, singing praises to your ability to make the unclear, clear. 
                          Until you got to the point where you seem to encourage a lack of transparency.



                          >At first, maybe this takes a little longer per story. This is not a real concern, as no one but the team really knows if the team
                          >can do twelve this Sprint or only ten. The team commits to what they can do. They do it, and the world becomes a better place.

                          This sounds like "lying to your product owner" (and the product owners that I've worked with, do know that "the team can do twelve")
                          This goes against the value of transparency. 
                          It also takes away the PO's right to incur short term technical debt in order to make a particular deadline.


                          One of the weaknesses of Scrum is that we think, it's self evident that product owners are responsible for maintaining the health their product while they provide value to their customer.

                          I believe that this all comes down to Stewardship, and not making it clear to the product owner that they need to be Stewards to their product.

                          I've got 3 questions, the answers to which I believe are currently "No, No and No"

                          1) Is there a Product Owner body where Product Owners discuss with other product owners how to be a better product owner?

                          2) Is the concept of Stewardship a part of the PO training?

                          3) Has the PO body taking on a commitment to maintaining and improving the health of their products?


                          Until the answer to the above 3 questions is Yes, Yes, and Yes, this is going to be an issue with Scrum.
                          Until we have some kind of "Product Owner Oath of Stewardship," this is going to be an issue with Scrum.
                          Until stewardship is self evident to people other than Scrum Masters, this is going to be an issue with Scrum.


                          Thanks

                          Malcolm Anderson
                          Scrum Coach & Agile Engineer





                          On Wed, Dec 1, 2010 at 6:16 AM, Ron Jeffries <ronjeffries@...> wrote:
                           

                          Hello, Roy. On Wednesday, December 1, 2010, at 3:29:27 AM, you
                          wrote:



                          > I think this notion of only ever doing things that have value to
                          > the customer has been somewhat misunderstood, or misapplied. There
                          > are most certainly things that need to be done that do not have an
                          > immediate, direct, or even obvious value to a customer ... but
                          > they still must be done. So what if user stories are not written
                          > for them, and so what if they really don't fit the notion of user
                          > stories (however defined). Being afraid to get it done because
                          > Scrum doesn't seem to cover that situation just leads to
                          > development paralysis.

                          If one is to do Scrum well, one needs to understand Scrum well.
                          Deviating from it every time something "seems" odd will not help a
                          team to prosper. It troubles me that so much of your advice, Roy,
                          deviates substantially from what Scrum and Agile teach, even when
                          there are perfectly good Scrum / Agile answers.

                          So here's so what.

                          The fundamental point of Scrum is for the PO to prioritize work
                          for the team, so that the best possible product can be delivered
                          within the desired time and budget. The PO will get the best
                          product if work is chosen that is of high "business value". (Low
                          development cost is also a concern in choosing the work, but may
                          will be substantially less important than value.)

                          Clearly, the PO is capable of driving out more value when a larger
                          percentage of the team's time comes under her control, since
                          otherwise, she is directing a smaller flow of work, which will
                          inherently reduce total value. Therefore it is undesirable to
                          slice away a strip of the team's development time and make it
                          unavailable for stories.

                          However, so-called "technical stories" have a value proposition
                          which is almost completely different from an ordinary story. A
                          technical refactoring story may promise faster velocity sometime
                          in the future, while an ordinary story is something that real
                          users actually want. It is therefore very difficult, perhaps
                          impossible, to compare an ordinary story and a technical story for
                          value. Therefore it is undesirable to present technical stories at
                          all, since they are so difficult to prioritize.

                          This seems, if one doesn't think about it very hard, to be a
                          dilemma. We can't take away the PO's time, we can't prioritize the
                          technical stories. We can't have them at all. We are doomed.

                          Like most dilemmas, this one is solved by using a third
                          alternative. It turns out that [almost?] every bit of technical
                          work, if it should be done at all, is done in support of one or
                          more stories. IN PARTICULAR, the kind of work the OP is talking
                          about, creation of tests and creation of clean code, is inherently
                          part of creating stories.

                          Why inherently, one might ask. Because for code to be DONE, it
                          must work, and to know whether it works, we must test it. Because
                          we are working incrementally, we will revisit much of the code
                          many times and we might break it, so it is wise that the tests be
                          recorded for reuse, i.e. automated.

                          Further, since we are building the system incrementally, we are
                          building the design incrementally. We do not have the time (nor,
                          likely, the ability) to define the design correctly up front. We
                          do not have the time to build it up front even if we did have the
                          ability to define it. We must, perforce, evolve the design.
                          Therefore, for each feature to be DONE, its design must be fit for
                          the current purpose.

                          Therefore, testing, and design improvement, are /inherently/ part
                          of the work of doing stories.

                          How does this help us with our dilemma? We have, through our human
                          failings, fallen behind on the testing and design. What are we to
                          do? We do not want to slice time away from the Product Owner. The
                          Product Owner cannot correctly prioritize these strange technical
                          stories. We are doomed.

                          No. All we need to do is to bear down a bit on improving the code
                          we work on, doing all it takes to bring it back up to the proper
                          definition of DONE, namely "well-tested and well-factored". We
                          make the code a bit better every time we pass through it.

                          At first, maybe this takes a little longer per story. This is not
                          a real concern, as no one but the team really knows if the team
                          can do twelve this Sprint or only ten. The team commits to what
                          they can do. They do it, and the world becomes a better place.

                          Better yet, soon, surprisingly soon, it doesn't take longer after
                          all. Since we are improving the code and tests in the areas we
                          work in, velocity picks up quite quickly in the areas we work in.
                          We wind up going faster than before, because we are going better.

                          Our Product Owner retains her ability to prioritize the whole of
                          our work, and our code improves. No confusion, no political
                          discussion, no need to make promises about the future that we may
                          not be able to keep.

                          We don't need technical stories or tricky negotiations. We just do
                          the job we were always supposed to do, namely make each story
                          DONE, fully tested, with a good design in place.


                          Ron Jeffries
                          www.XProgramming.com
                          Thousands of years ago, the first man discovered how to make fire.
                          He was probably burned at the stake he had taught his brothers to
                          light - Howard Roark (The Fountainhead, Ayn Rand)





                        • banshee858
                          ... Have you tried something like this? List all the technical debt stuff on little post-it notes adjacent to the Task Board. When a Product Backlog item
                          Message 12 of 28 , Dec 2, 2010
                            --- In scrumdevelopment@yahoogroups.com, Paul Tevis <ptevis@...> wrote:
                            >
                            > The team I'm on right now is about four months into a Scrum
                            > transition, and we're finally starting to deal with technical debt
                            > reduction and technical practice improvement. One thing that I'm
                            > struggling with is how to make the work that we're doing on these
                            > visible to the team, while at the same time keeping our focus on
                            > delivering customer value (i.e. doing real stories).
                            >
                            > My concerns are (1) as soon as we start tracking non-story tasks we'll
                            > lose focus on delivering customer value, and (2) if we don't make
                            > these sorts of tasks visible, we won't make progress on them at the
                            > rate we need to. What are good patterns you've seen for dealing with
                            > technical tasks that aren't directly attached to a story (or that cut
                            > across multiple stories)?
                            >
                            Have you tried something like this?

                            List all the technical debt stuff on little post-it notes adjacent to the Task Board. When a Product Backlog item (PBI) is selected for a Sprint, look at the technical debt pieces and find the pieces that make sense to finish while working on the PBI. Add those to the scope of work for the PBI and estimate how long it will take to complete the feature and the technical debt pieces.

                            That way you make the technical debt work visible, you can prioritize it and link it to real value.

                            Carlton
                          • Ron Jeffries
                            Hello, banshee858. On Thursday, December 2, 2010, at 2:20:45 PM, ... Very nice! Ron Jeffries www.XProgramming.com We accomplish what we understand. If we are
                            Message 13 of 28 , Dec 2, 2010
                              Hello, banshee858. On Thursday, December 2, 2010, at 2:20:45 PM,
                              you wrote:

                              > Have you tried something like this?

                              > List all the technical debt stuff on little post-it notes
                              > adjacent to the Task Board. When a Product Backlog item (PBI) is
                              > selected for a Sprint, look at the technical debt pieces and find
                              > the pieces that make sense to finish while working on the PBI.
                              > Add those to the scope of work for the PBI and estimate how long
                              > it will take to complete the feature and the technical debt pieces.

                              > That way you make the technical debt work visible, you can
                              > prioritize it and link it to real value.

                              Very nice!

                              Ron Jeffries
                              www.XProgramming.com
                              We accomplish what we understand. If we are to accomplish something
                              together, we need to understand it together.
                            • Adam Sroka
                              On Wed, Dec 1, 2010 at 6:39 PM, Malcolm Anderson ... If you were going to make that business decision you would at least want to know: 1) How much am I
                              Message 14 of 28 , Dec 2, 2010
                                On Wed, Dec 1, 2010 at 6:39 PM, Malcolm Anderson
                                <malcolm.b.anderson@...> wrote:
                                >
                                >
                                > There are a lot of reasons for taking on short term
                                > technical debt. I use the analogy of the credit card with a 36%
                                > interest rate. You do not want to use that card. But there are times
                                > when it makes great business sense to use that card in a short term
                                > situation.
                                >

                                If you were going to make that business decision you would at least
                                want to know:

                                1) How much am I spending?

                                2) How much interest will I incur? Over what period of time?

                                3) Will I have enough revenue to pay it back?

                                When teams voluntarily take on technical debt they rarely (if ever)
                                know the answer to even one of these questions.

                                The interest we are talking about is much higher than you think. It is
                                more analogous to a loan shark than a credit card. I have seen teams
                                create enough of a mess in a single two-week Sprint that they were
                                never able to fix it. I myself am only able to fix the messes I make
                                with a reasonable amount of effort if I do it within a few days. After
                                a few weeks the effort is an order of magnitude higher than the
                                initial investment.

                                < beating a dead horse >
                                This is not what the technical debt metaphor was ever intended to
                                address. Even when we do the best job possible our understanding of
                                the problem evolves over time. We have to go back periodically and
                                refactor the code to represent our *current* understanding of the
                                problem. If we don't then our understanding and the code will continue
                                to diverge until the code becomes very hard to maintain. *That* is
                                what technical debt means. Neither Ward Cunningham (the originator of
                                the metaphor) nor anyone who understood him ever said that writing
                                crappy, untested code is sometimes a good business decision.
                                < / beating dead horse >
                              • Alan Dayley
                                +10 Alan
                                Message 15 of 28 , Dec 2, 2010
                                  +10

                                  Alan

                                  On Thu, Dec 2, 2010 at 1:56 PM, Adam Sroka <adam.sroka@...> wrote:
                                   

                                  On Wed, Dec 1, 2010 at 6:39 PM, Malcolm Anderson

                                  > There are a lot of reasons for taking on short term
                                  > technical debt. I use the analogy of the credit card with a 36%
                                  > interest rate. You do not want to use that card. But there are times
                                  > when it makes great business sense to use that card in a short term
                                  > situation.
                                  >

                                  If you were going to make that business decision you would at least
                                  want to know:

                                  1) How much am I spending?

                                  2) How much interest will I incur? Over what period of time?

                                  3) Will I have enough revenue to pay it back?

                                  When teams voluntarily take on technical debt they rarely (if ever)
                                  know the answer to even one of these questions.

                                  The interest we are talking about is much higher than you think. It is
                                  more analogous to a loan shark than a credit card. I have seen teams
                                  create enough of a mess in a single two-week Sprint that they were
                                  never able to fix it. I myself am only able to fix the messes I make
                                  with a reasonable amount of effort if I do it within a few days. After
                                  a few weeks the effort is an order of magnitude higher than the
                                  initial investment.

                                  < beating a dead horse >
                                  This is not what the technical debt metaphor was ever intended to
                                  address. Even when we do the best job possible our understanding of
                                  the problem evolves over time. We have to go back periodically and
                                  refactor the code to represent our *current* understanding of the
                                  problem. If we don't then our understanding and the code will continue
                                  to diverge until the code becomes very hard to maintain. *That* is
                                  what technical debt means. Neither Ward Cunningham (the originator of
                                  the metaphor) nor anyone who understood him ever said that writing
                                  crappy, untested code is sometimes a good business decision.
                                  < / beating dead horse >


                                • banshee858
                                  ... Not my idea - it came from Tobias Mayer. Carlton
                                  Message 16 of 28 , Dec 3, 2010
                                    >
                                    > > List all the technical debt stuff on little post-it notes
                                    > > adjacent to the Task Board. When a Product Backlog item (PBI) is
                                    > > selected for a Sprint, look at the technical debt pieces and find
                                    > > the pieces that make sense to finish while working on the PBI.
                                    > > Add those to the scope of work for the PBI and estimate how long
                                    > > it will take to complete the feature and the technical debt pieces.
                                    >
                                    > > That way you make the technical debt work visible, you can
                                    > > prioritize it and link it to real value.
                                    >
                                    > Very nice!
                                    >
                                    Not my idea - it came from Tobias Mayer.

                                    Carlton
                                  • tobias.mayer
                                    I can t take credit for it either. I first witnessed this with a team I worked with at Business Week in NYC. It emerged from the collective consciousness of
                                    Message 17 of 28 , Dec 6, 2010
                                      I can't take credit for it either. I first witnessed this with a team I worked with at Business Week in NYC. It emerged from the collective consciousness of the team. As I recall this is what they did (just expanding on Carlton's description):

                                      Whenever they found technical debt, or any kind of code problem that needed fixing, they didn't fix it unless it was essential to complete the story they were working on (this took some discipline). Otherwise they wrote a sticky note (a task) and added it to a special box on the visual management wall labeled Technical Debt. This served two purposes: a reminder to the team, and a visual representation to the business of the state of the codebase.

                                      At any subsequent planning meeting, when the USER-FACING stories were discussed (I don't buy into the concept of "technical stories") someone may remind the team that if such-and-such a story was attempted then certain code debt items called out on the wall would need to be fixed as part of that story. The rule was "no workarounds". This clean up work was then considered as part of the essential work of the story, and the story was estimated accordingly.

                                      These tasks were NOT part of the backlog, they didn't get prioritized and were never fixed in isolation. Instead, when the story was estimated and committed to, these tasks simply became extra tasks towards the completion of the story. It was very elegant, very simple and it worked well for that team. Code debt got cleaned up when doing so resulted in value to the user.

                                      And visually everyone could see the results of that clean up: An fast-emptying space on the wall.

                                      Tobias


                                      --- In scrumdevelopment@yahoogroups.com, "banshee858" <cnett858@...> wrote:
                                      >
                                      > >
                                      > > > List all the technical debt stuff on little post-it notes
                                      > > > adjacent to the Task Board. When a Product Backlog item (PBI) is
                                      > > > selected for a Sprint, look at the technical debt pieces and find
                                      > > > the pieces that make sense to finish while working on the PBI.
                                      > > > Add those to the scope of work for the PBI and estimate how long
                                      > > > it will take to complete the feature and the technical debt pieces.
                                      > >
                                      > > > That way you make the technical debt work visible, you can
                                      > > > prioritize it and link it to real value.
                                      > >
                                      > > Very nice!
                                      > >
                                      > Not my idea - it came from Tobias Mayer.
                                      >
                                      > Carlton
                                      >
                                    • Ron Jeffries
                                      Hello, Tobias, Excellent! Just what I like to see! R On Monday, December 6, 2010, at 1:51:46 PM, ... Ron Jeffries www.XProgramming.com In times of stress, I
                                      Message 18 of 28 , Dec 6, 2010
                                        Hello, Tobias,

                                        Excellent! Just what I like to see!

                                        R

                                        On Monday, December 6, 2010, at 1:51:46 PM,
                                        you wrote:

                                        > I can't take credit for it either. I first witnessed this with a
                                        > team I worked with at Business Week in NYC. It emerged from the
                                        > collective consciousness of the team. As I recall this is what
                                        > they did (just expanding on Carlton's description):

                                        > Whenever they found technical debt, or any kind of code problem
                                        > that needed fixing, they didn't fix it unless it was essential to
                                        > complete the story they were working on (this took some
                                        > discipline). Otherwise they wrote a sticky note (a task) and added
                                        > it to a special box on the visual management wall labeled
                                        > Technical Debt. This served two purposes: a reminder to the team,
                                        > and a visual representation to the business of the state of the codebase.

                                        > At any subsequent planning meeting, when the USER-FACING stories
                                        > were discussed (I don't buy into the concept of "technical
                                        > stories") someone may remind the team that if such-and-such a
                                        > story was attempted then certain code debt items called out on the
                                        > wall would need to be fixed as part of that story. The rule was
                                        > "no workarounds". This clean up work was then considered as part
                                        > of the essential work of the story, and the story was estimated accordingly.

                                        > These tasks were NOT part of the backlog, they didn't get
                                        > prioritized and were never fixed in isolation. Instead, when the
                                        > story was estimated and committed to, these tasks simply became
                                        > extra tasks towards the completion of the story. It was very
                                        > elegant, very simple and it worked well for that team. Code debt
                                        > got cleaned up when doing so resulted in value to the user.

                                        > And visually everyone could see the results of that clean up: An fast-emptying space on the wall.



                                        Ron Jeffries
                                        www.XProgramming.com
                                        In times of stress, I like to turn to the wisdom of my Portuguese waitress,
                                        who said: "Olá, meu nome é Marisol e eu serei sua garçonete."
                                        -- after Mark Vaughn, Autoweek.
                                      Your message has been successfully submitted and would be delivered to recipients shortly.