Loading ...
Sorry, an error occurred while loading the content.

Re: [scrumdevelopment] Technical Debt Measurement

Expand Messages
  • Kurt Häusler
    ... were an airplane would you fly in it? sounds more like the existence of known bugs to me, which is, as far as I know, different to technical debt.
    Message 1 of 14 , Mar 1, 2011
    • 0 Attachment
      On Wed, Mar 2, 2011 at 7:14 AM, Peter Stevens (cal) <peterstev@...> wrote:

      As a Product Owner, ask your team at the sprint review 'Can we release it' (or more to the point, 'If it were an airplane would you fly in it?' If the answer is anything less than an unqualified 'Yes', then ask why? 'Uh, we need to do integration' 'We need to do regression tests' - all the explanations are unfinished work and need to come into the backlog, so that they will be done before release. This unfinished work needs to be completed before a release can take place. This is also technical debt.

      Although software with technical debt is still perfectly releasable. "If it were an airplane would you fly in it?" sounds more like the existence of known bugs to me, which  is, as far as I know, different to technical debt. Needing to do integration or regression tests doesn't sound like technical debt either, it is as you say unfinished work, but is that really technical debt?

      Technical debt to me is like a big ball of mud that perfectly fulfills the customer requirements with no known bugs. If the work is unfinished, or does not fulfill the requirements, or has known bugs, then they should all be looked at before technical debt. Technical debt is the duplicated code, the poorly named, or over-sized methods, the business logic mixed in with the presentation layer, or the data access mixed in with the business logic. It is the case statements, or if chains, that should be refactored into polymorphic classes. It is the dead code that never gets called, or the commented out code, the classes that implement interfaces, yet only some of the interface methods are relevant.

      Generally technical debt only affects the customer through higher maintenance costs.

      Probably better questions to ask would be "are you proud of it?" or "would uncle Bob approve?"
    • Matthew
      ... There are a bunch of measurements for code: Cyclomatic Complexity, CrapForJ, etc. That said, overall, I am more interested in /qualitative assessment/ over
      Message 2 of 14 , Mar 3, 2011
      • 0 Attachment
        --- In scrumdevelopment@yahoogroups.com, Geetha Anand <reachgeethaanand@...> wrote:
        >
        > HI,
        >
        > I would like to know how Technical Debt is usually measured and reported
        > and how it is used.
        >
        > Thanks
        > Geetha
        >

        There are a bunch of measurements for code: Cyclomatic Complexity, CrapForJ, etc.

        That said, overall, I am more interested in /qualitative assessment/ over /quantiative/.

        In other words, I'm more interested in the story over some number.

        The first book is out in this field - "Managing Software Debt" by Chris Sterling; I did an interview with him last week:

        http://www.informit.com/articles/article.aspx?p=1684785

        I would be /*VERY*/ reluctant to reduce something as complex as tech debt to a single number, or even a balanced scorecard.

        For a longer discussion about using measurements in software, please allow me to recommend "Software Engineering Metrics: What do they measure and how do we know?"

        http://www.kaner.com/pdfs/metrics2004.pdf

        regards,



        ---------- Forwarded message ----------
        From: Matt Heusser <matt.heusser@...>
        Date: Tue, Mar 1, 2011 at 8:54 AM
        Subject: Ready for your feedback: Best SW Test Writing
        To: Walker E Royce <weroyce@...>, james bach <james@...>, Bernie Berger <bernie@...>, "Gregory M. Pope" <pope12@...>


        I've completed typing in your chapters into Best SW Test Writing and have created an introduction.

        The next step is to review your chapters and create a "What I have learned since" section at the bottom.

        (A) You may disagree with how I characterized the introduction. If you do, drop me a line

        (B) You might not be interested in writing "what I have learned since" (Greg's article is pretty new and complete) or want to characterize it differently. If you do, just drop me a note, I'm flexible.

        If you can review and add a postscript to your chapter, I have everything I'll need to send the proposal to the publisher.

        If you can't get to it within two weeks, let me know, i'll submit without it and you can make changes by -- say -- June 1st?

        Oh, I move fast.

        Chapters:

        Walker/Winston Royce:
        Managing the development of Large Software Systems

        Greg Pope:
        "Test" is a Four Letter Word

        James Bach:
        Software Test Automation Snake Oil ("what I learned since" /is/ the follow-up chapter)
        Agile Test Automation

        Bernie Berger:
        Efficient Testing with All Pairs


        regards,


        --
        Matthew Heusser,
        Personal Blog: http://xndev.blogspot.com/
        Test Community Blog: http://softwaretestpro.com/blog/
        Twitter: mheusser
      • Charles Bradley - Scrum Coach CSM PSM I
        Geetha, Just in case this is not obvious, Scrum(as defined in The Scrum Guide) does not cover the topic of Technical Debt directly. I imagine many people have
        Message 3 of 14 , Mar 3, 2011
        • 0 Attachment
          Geetha,

          Just in case this is not obvious, Scrum(as defined in The Scrum Guide) does not cover the topic of Technical Debt directly.

          I imagine many people have strategies that they use wrt Technical Debt and Scrum.  Were you asking how people view/handle TD with respect to Scrum?
           
          -------
          Charles Bradley, CSM, PSM I
          Experienced Scrum Coach
          My blog: http://scrumcrazy.wordpress.com/



          From: Geetha Anand <reachgeethaanand@...>
          To: scrumdevelopment@yahoogroups.com
          Sent: Tue, March 1, 2011 9:29:58 PM
          Subject: [scrumdevelopment] Technical Debt Measurement



          HI,
           
          I would like to know how Technical Debt is usually measured  and reported and how it is used.
           
          Thanks
          Geetha
           


        • JackM
          The impact of technical debt is that it slows teams down. So I think velocity would be one way to measure this. If your velocity starts tanking then you know
          Message 4 of 14 , Mar 3, 2011
          • 0 Attachment
            The impact of technical debt is that it slows teams down. So I think velocity would be one way to measure this. If your velocity starts tanking then you know you have have problems which might be technical debt. Closure phase starts taking longer and longer.

            Jack
            www.agilebuddy.com

            --- In scrumdevelopment@yahoogroups.com, Geetha Anand <reachgeethaanand@...> wrote:
            >
            > HI,
            >
            > I would like to know how Technical Debt is usually measured and reported
            > and how it is used.
            >
            > Thanks
            > Geetha
            >
          • Mark Levison
            ... Sadly that s trailing measure :-) By the time it tells you anything you have troubl. Tools like Sonar (from Codehaus) claim to provide a numerical measure.
            Message 5 of 14 , Mar 3, 2011
            • 0 Attachment
              On Thu, Mar 3, 2011 at 8:31 PM, JackM <jack@...> wrote:
               

              The impact of technical debt is that it slows teams down. So I think velocity would be one way to measure this. If your velocity starts tanking then you know you have have problems which might be technical debt. Closure phase starts taking longer and longer.


              Sadly that's trailing measure :-) By the time it tells you anything you have troubl.

              Tools like Sonar (from Codehaus) claim to provide a numerical measure. Its a starting point if you understand how it works and take it with a large grain of salt.

              Cheers
              Mark Levison

              MarkMark Levison | Agile Pain Relief Consulting | Certified Scrum Trainer
              Agile Editor @ InfoQ | Blog | Twitter | Office: (613) 862-2538
              Recent Entries:
              Story Slicing How Small is Small Enough, Why use an Agile Coach


            • Adam Sroka
              There seem to be multiple, diverging definitions of technical debt. This is disturbing, because it makes it a lot harder to talk about, let alone measure. As
              Message 6 of 14 , Mar 3, 2011
              • 0 Attachment
                There seem to be multiple, diverging definitions of technical debt. This is disturbing, because it makes it a lot harder to talk about, let alone measure. 

                As some of you may know, Ward's original definition of technical debt was meant to reflect the fact that we are driving towards a moving target. We build the best system we know how to today, and nonetheless as our understanding of the domain evolves it diverges from what we wrote in the code. Ward suggests that we should refactor the code to reflect our /current/ understanding, and that if we don't we will be slowed down by this divergence. 

                This isn't what most people seem to mean by technical debt, though. Most people seem to be saying that anything undesirable about our design which could be refactored is technical debt. This is a potentially useful definition, and we could even measure this with some of the design metrics that others have talked about (Although no design metric perfectly reflects the quality of a design. There is always a subjective element to it.) 

                Despite its usefulness, this definition becomes troubling to me when it is used to justify making questionable design decisions now in favor of refactoring later (In effect, borrowing against our technical credit. We haven't quite beat the metaphor to death at this point.) I assert that it is a good idea to always create the simplest design we know how to. It takes considerable skill to make poor design decisions now and then be able to fix them later without spending much (much) more than we would have by just doing the simplest thing in the first place. 

                What is perhaps a bit more troubling to me is that I am now hearing a third potential definition for technical debt: any technical choice which reduces our velocity. This is actually a useful idea, but it becomes difficult to work with when we consider all of the different kinds of technical choices that can have an effect. 

                Is language choice a kind of tech debt? Is framework choice? Is database choice? Certainly I can have a much higher velocity if I create my simple data-driven web app using Rails than if I create the same app in COBOL (With Rails much of what I need is generated for me by convenient rake tasks. With COBOL I have to compile and link just to run a single unit test.) 

                We could come up with separate terms for each of these things, but I think it makes more sense to consider something like Theory of Constraints for managing this. If I need to be able to go faster then I should look for the biggest impediment to that first. It might be that language or framework choices are slowing me down. It might be that poor design is slowing me down. Or, it might just be that I am putting off refactoring longer than I really should. 

                On Thu, Mar 3, 2011 at 5:31 PM, JackM <jack@...> wrote:
                 

                The impact of technical debt is that it slows teams down. So I think velocity would be one way to measure this. If your velocity starts tanking then you know you have have problems which might be technical debt. Closure phase starts taking longer and longer.

                Jack
                www.agilebuddy.com



                --- In scrumdevelopment@yahoogroups.com, Geetha Anand <reachgeethaanand@...> wrote:
                >
                > HI,
                >
                > I would like to know how Technical Debt is usually measured and reported
                > and how it is used.
                >
                > Thanks
                > Geetha
                >


              • JackM
                I believe Ken Schwabber in his google tech talk talked about technical debt and in particular made reference to Core or Infrastructure software where
                Message 7 of 14 , Mar 3, 2011
                • 0 Attachment
                  I believe Ken Schwabber in his google tech talk talked about technical debt and in particular made reference to "Core or Infrastructure software" where essentially new functionality is throttled by the the core software, core software basically, riddled with technical debt, un-refactored code etc. so Velocity in the core is way lower. Also references Design Dead software!

                  Check it out...

                  http://video.google.com/videoplay?docid=-7230144396191025011#

                  jack
                  www.agilebuddy.com

                  --- In scrumdevelopment@yahoogroups.com, Adam Sroka <adam.sroka@...> wrote:
                  >
                  > There seem to be multiple, diverging definitions of technical debt. This is
                  > disturbing, because it makes it a lot harder to talk about, let alone
                  > measure.
                  >
                  > As some of you may know, Ward's original definition of technical debt was
                  > meant to reflect the fact that we are driving towards a moving target. We
                  > build the best system we know how to today, and nonetheless as our
                  > understanding of the domain evolves it diverges from what we wrote in the
                  > code. Ward suggests that we should refactor the code to reflect our
                  > /current/ understanding, and that if we don't we will be slowed down by this
                  > divergence.
                  >
                  > This isn't what most people seem to mean by technical debt, though. Most
                  > people seem to be saying that anything undesirable about our design which
                  > could be refactored is technical debt. This is a potentially useful
                  > definition, and we could even measure this with some of the design metrics
                  > that others have talked about (Although no design metric perfectly reflects
                  > the quality of a design. There is always a subjective element to it.)
                  >
                  > Despite its usefulness, this definition becomes troubling to me when it is
                  > used to justify making questionable design decisions now in favor of
                  > refactoring later (In effect, borrowing against our technical credit. We
                  > haven't quite beat the metaphor to death at this point.) I assert that it is
                  > a good idea to always create the simplest design we know how to. It takes
                  > considerable skill to make poor design decisions now and then be able to fix
                  > them later without spending much (much) more than we would have by just
                  > doing the simplest thing in the first place.
                  >
                  > What is perhaps a bit more troubling to me is that I am now hearing a third
                  > potential definition for technical debt: any technical choice which reduces
                  > our velocity. This is actually a useful idea, but it becomes difficult to
                  > work with when we consider all of the different kinds of technical choices
                  > that can have an effect.
                  >
                  > Is language choice a kind of tech debt? Is framework choice? Is database
                  > choice? Certainly I can have a much higher velocity if I create my simple
                  > data-driven web app using Rails than if I create the same app in COBOL (With
                  > Rails much of what I need is generated for me by convenient rake tasks. With
                  > COBOL I have to compile and link just to run a single unit test.)
                  >
                  > We could come up with separate terms for each of these things, but I think
                  > it makes more sense to consider something like Theory of Constraints for
                  > managing this. If I need to be able to go faster then I should look for the
                  > biggest impediment to that first. It might be that language or framework
                  > choices are slowing me down. It might be that poor design is slowing me
                  > down. Or, it might just be that I am putting off refactoring longer than I
                  > really should.
                  >
                  > On Thu, Mar 3, 2011 at 5:31 PM, JackM <jack@...> wrote:
                  >
                  > >
                  > >
                  > > The impact of technical debt is that it slows teams down. So I think
                  > > velocity would be one way to measure this. If your velocity starts tanking
                  > > then you know you have have problems which might be technical debt. Closure
                  > > phase starts taking longer and longer.
                  > >
                  > > Jack
                  > > www.agilebuddy.com
                  > >
                  > >
                  > > --- In scrumdevelopment@yahoogroups.com, Geetha Anand <reachgeethaanand@>
                  > > wrote:
                  > > >
                  > > > HI,
                  > > >
                  > > > I would like to know how Technical Debt is usually measured and reported
                  > > > and how it is used.
                  > > >
                  > > > Thanks
                  > > > Geetha
                  > > >
                  > >
                  > >
                  > >
                  >
                • Alan Dayley
                  Be careful what you measure. Doing so may not provide what you think or will have damaging side effects. What is the benefit or benefits you are seeking by
                  Message 8 of 14 , Mar 3, 2011
                  • 0 Attachment
                    Be careful what you measure.  Doing so may not provide what you think or will have damaging side effects.

                    What is the benefit or benefits you are seeking by measuring technical debt?

                    Who is the customer for this information?

                    What will the customer do with the information?

                    Alan

                    On Tue, Mar 1, 2011 at 9:29 PM, Geetha Anand <reachgeethaanand@...> wrote:
                     

                    HI,
                     
                    I would like to know how Technical Debt is usually measured  and reported and how it is used.
                     
                    Thanks
                    Geetha
                     


                  • Matthew
                    ... I think it s important to note that tech debt is a metaphor, a thought experiment, not an actual like, thing. So any hard attempt to measure by reducing
                    Message 9 of 14 , Mar 4, 2011
                    • 0 Attachment
                      --- In scrumdevelopment@yahoogroups.com, Adam Sroka <adam.sroka@...> wrote:
                      >
                      > There seem to be multiple, diverging definitions of technical debt. This is
                      > disturbing, because it makes it a lot harder to talk about, let alone
                      > measure.
                      >

                      I think it's important to note that tech debt is a metaphor, a thought experiment, not an actual like, thing. So any hard attempt to 'measure' by reducing to a number is going to be measure some attribute (like velocity or complexity), not the /actual/ debt.

                      Because there is no actual debt; it is a /concept/. To attempt to make real what is not is a sort of reification fallacy -- which, I feel obligated to add, is the same thing we do with velocity and complexity.

                      (In other words, whatever use use to measure velocity isn't actually velocity either, it's some approximation.)

                      > As some of you may know, Ward's original definition of technical debt was
                      > meant to reflect the fact that we are driving towards a moving target. We
                      > build the best system we know how to today, and nonetheless as our
                      > understanding of the domain evolves it diverges from what we wrote in the
                      > code. Ward suggests that we should refactor the code to reflect our
                      > /current/ understanding, and that if we don't we will be slowed down by this
                      > divergence.
                      >

                      I had to "well actually" you, but I think when Ward introduced the metaphor he was speaking to the issue of deferring some improvement in order to hit a deadline. You can think of this as sort of "borrowing trouble" - it allows you to go faster /now/ but will cause some penalty in the future, just like real debt. (In addition to refactoring, I think one of the classic examples is failing to upgrade the compiler from c++ to dotnet in 2002; "we'll do that next release. It would take a couple days we don't have right now.")

                      Now there are responsible ways to manage real debt. For example, you might borrow money in order to build a factory, with a plan of making money faster than the interest accrued on the debt.

                      What you don't want to do is borrow money without realizing you are borrowing money -- which is exactly what happens every time a manager says "JUST HIT THE DEADLINE!" and people make irresponsible compromises in order to hit the deadline.

                      So the Tech Debt metaphor can be useful in explaining those tradeoffs to executives, to "speak in their language."

                      I feel obligated to add that there is a difference between consciously taking on tech debt and writing crappy code; writing crappy code is just bad.

                      In general, though, once we have this conversation, people want to /measure/.

                      They want to get out cyclomatic complexity, or take the first derivative of velocity (which is acceleration), or other stuff like that.

                      The problem is, once you start measuring and controling, you introduce dysfunction. At the same time, since most of our metrics are either subjective proxies (velocity), or objective proxies (complexity, lines of code), the *numbers* tend to will go up, but, over time, the difference between what you are actually measuring and the desired change becomes greater and greater.

                      In the end, I don't recommend quantitative measurement of a metaphor. It doesn't make any sense, and Steve Poling and I dropped our book project. (After serving as the organizers of the Agile Alliance Workshop on the subject.)


                      > Despite its usefulness, this definition becomes troubling to me when it is
                      > used to justify making questionable design decisions now in favor of
                      > refactoring later (In effect, borrowing against our technical credit. We
                      > haven't quite beat the metaphor to death at this point.)
                      >

                      What I'm saying here is that if you make the decision consciously, transparently, with management informed of the consequence, it might be just fine.

                      *That* said, it is *not* okay to do crappy work and blame management. Like a doctor who said his patient "wasn't worried" about him washing his hands and he "needed treatment right now", as professionals we have a responsibility to do work that is fundamentally good. Where we draw that line is a tough discussion.

                      My short answer is that the team should have some shared agreement on minimum standards for the code, and /every/ change to the system should be at that level or higher.

                      Bob Martin has a book, "Clean Coder", coming out in a few months that discusses that issue in more depth.

                      I think I'll stop rambling now. :-)

                      regards,



                      --
                      Matthew Heusser,
                      Personal Blog: http://xndev.blogspot.com/
                      Test Community Blog: http://softwaretestpro.com/blog/
                      Twitter: mheusser
                    • Charles Bradley - Scrum Coach CSM PSM I
                      I concur that the technical debt term does not have one well accepted definition. While velocity can go down due to technical debt, it can also go up, if the
                      Message 10 of 14 , Mar 4, 2011
                      • 0 Attachment
                        I concur that the technical debt term does not have one well accepted definition.

                        While velocity can go down due to technical debt, it can also go up, if the team decides to assign story points to technical debt (which is generally a big no no, even in most of the accepted definitions of TD).

                        I wouldn't place a lot of stock in trying to correlate velocity and TD.  If people measured velocity and story points the way they were intended, AND if they didn't inflate their story points to reflect technical debt, then maybe a lower velocity could be explained by TD, but it could also be explained by a whole host of other things(staff turnover, staff reduction, corporate restructuring, etc).

                        In short, I wouldn't bother trying to relate the two.

                        If I wanted to measure technical debt, I think I would ask the team to first give me their views on what they consider TD in the system, and how much they feel it affects new features and/or maintenance costs.  After that, I would also ask them to use some tools to measure code quality like Sonar does.  From those two viewpoints, I think I'd have a pretty good idea how much TD the system had in it.  OTOH, one man's "TD" is another man's "no biggie", so whoever is making the assessment will have to put some subjectivity into the situation.
                         
                        -------
                        Charles Bradley, CSM, PSM I
                        Experienced Scrum Coach
                        My blog: http://scrumcrazy.wordpress.com/



                      • Ron Jeffries
                        Hello, Adam. I think you raise a good point here. It would be good to have terms for the various kinds of what we now call technical debt to help us talk
                        Message 11 of 14 , Mar 4, 2011
                        • 0 Attachment
                          Hello, Adam.

                          I think you raise a good point here. It would be good to have terms
                          for the various kinds of what we now call "technical debt" to help
                          us talk about them more clearly. Some random thoughts:

                          1. I believe you are correct about Ward's original usage meaning
                          a gap between our current understanding and the code's
                          understanding of the domain and the suitable design.

                          I also believe that I did not understand that until his recent
                          writings on the subject.

                          2. Then there is "anything undesirable about the design". I agree
                          that this is important, and I think that we'll find that there
                          are some good things we can say about it, and that some of it
                          will elude us and require us to clarify our thinking. I agree
                          that some important things remain subjective and I believe they
                          always will. At the same time, some very interesting metrics are
                          out there.

                          3. With you, I am very concerned about the notion of making
                          questionable design decisions in the name of speeding up. We have
                          probably all tried this, and gotten away with it, in the sense
                          that we shipped our code. This does not really tell us anything
                          about whether it was a good idea: it only tells us that we did it
                          and survived.

                          Some name brand people, notably Joshua and Jim Shore, have been
                          saying that you can squeeze code quality and get some time out of
                          it. I think this is not proven, even for them, and that their
                          idea of squeezing code quality and that of their readers are
                          probably quite different. I would bet that they don't have a lot
                          of 400 line methods even in their worst code.

                          4. You mention also other technical choices affecting velocity.
                          As you say, language, framework, database, all these things
                          clearly affect velocity. I would not be inclined to include them
                          in technical debt offhand but I can certainly understand someone
                          feeling that some legacy decision had messed them over. A
                          decision to use C++ for example. Or Java. :)

                          > We could come up with separate terms for each of these things, but I think
                          > it makes more sense to consider something like Theory of Constraints for
                          > managing this. If I need to be able to go faster then I should look for the
                          > biggest impediment to that first. It might be that language or framework
                          > choices are slowing me down. It might be that poor design is slowing me
                          > down. Or, it might just be that I am putting off refactoring longer than I
                          > really should.

                          Yes. I agree that we need to look at the current real impediments.
                          That said, there is, it seems to me, one very common thread, which
                          is that many teams create designs which, rather than improving over
                          time, get worse. Whether we call that technical debt or code rot,
                          it's a big problem.

                          Ron Jeffries
                          www.XProgramming.com
                          Example isn't another way to teach, it is the only way to teach.
                          --Albert Einstein
                        Your message has been successfully submitted and would be delivered to recipients shortly.