Loading ...
Sorry, an error occurred while loading the content.

Re: [scrumdevelopment] Re:Sprint Quality Metric

Expand Messages
  • Eric Deslauriers
    I m quite surprised to see proposals that tracking defects = wasting time . What metrics are you using to judge how good the delivery was to Prod? What metric
    Message 1 of 22 , Nov 12, 2008
      I'm quite surprised to see proposals that "tracking defects = wasting time".

      What metrics are you using to judge how good the delivery was to Prod? What metric is the team being told they're targeting? 

      For the age metric, I prefer a variant (which I'm sure you're also using) - age of the oldest "unresolved" P1 defect, then P2, then... The customer, and I, would rather the team invest time in a P1 or P2 than a 25 day old P4. However, we can't release without managing all open defects, including that P4 and making a decision about it.

      A valid resolution is "We're not going to fix this ever."
      Another one is "We've decided to leak with this defect and will fix it in version P.Q.R"
      Both require documentation delivered to the customer to explain the issue and the workaround(s) available, if any (almost always).

      My objective is not perfect code, my objective is high quality, clear expectations of the issues we are choosing to leak because they don't detract from the ROI of the product to the customer in any significant way. It's about business value, after all.

      Tracking defects allows the team to make a decision, rather than the individuals. It also helps explain why our velocity is X or Y. Lots of defects going into the test and integration phases (notice I'm not saying "delivered to the QA team"), as indicated by lots of defects found during the test phase, says we need to revisit our development processes.

      One metric we use for quality is 5% of defects leaked to Prod vs found in the implementation/test phases, and no more than a few P1/P2 issues. Hmm. Now it's important to track defects in implementation. And the team has internal standards for what constitutes a countable defect and is pretty allergic to defects for defect's sake. All metrics can be worked. Give me any metric, I can work it.

      Agile is about the spirit of the need, not the letter of the need (interactions over contracts).

      Another option is 0.5 defects per new/modified KLOC, when you have KLOC to count  - lots of dev envs today are graphical, XML-based, configuration, etc. How do you count defects/KLOC when the team misconfigured the java appliance, Oracle Apps, DB, etc.? It's a defect, that's for sure. But you can't count KLOC in the same way (and not all tools count it the same way...).

      I don't use the # to judge individuals, but there are those infrequent occasions the team is glad I am using #s to drive a conversation with a specific member of the team. Always, of course, taking in the complexity of the space, the newness of the tech, the history of the code (has it been around a long time and is mostly stable, is it smoking new and we were date driven, etc.).

      We also limit discussion about whether it's a defect or not to 5 minutes, to put the kibosh on those who want to battle it "tooth and nail" - a highly unproductive behaviour and one we have no room for in our teams. If you can't convince the person who thinks it's a defect that it's not in 5 minutes, it is a defect and the PO, as the wringable neck, can make the final call as needed.

      So... What metrics does your team use to judge quality and degree of success of a deliverable??

      Eric D

      On Fri, Oct 24, 2008 at 4:16 AM, Ilja Preuß <iljapreuss@...> wrote:

      2008/10/24 Keith Sterling <keith@...>:

      > If you have continuous integration platform then this can be used to provide
      > a huge range of metrics

      Yes. The problem being, of course, that you then have a huge range of
      metrics, but no means of interpreting them consistently and
      productively. Better to have a small, carefully choosen range of
      metrics, based on what you are actually interested in achieving.

      A metric that I find interesting as an indicator of the quality of the
      process for dealing with bugs is "age of the oldest open bug".

      Cheers, Ilja

      Eric D
      08 K1200S Tricolor (phreowww)
      06 Husqvarna TE610
    Your message has been successfully submitted and would be delivered to recipients shortly.