Loading ...
Sorry, an error occurred while loading the content.

Re: [scrumdevelopment] How to measure the value prod uced by a devops team / team kpi´s!?

Expand Messages
  • Wouter Lagerweij
    Hi Daniele, Interesting topic! What is the main reason your team feels you need to emphasize your team s value to the rest of the organisation? When using
    Message 1 of 4 , May 9, 2012
    • 0 Attachment
      Hi Daniele,

      Interesting topic!

      What is the main reason your team feels you need to emphasize your team's value to the rest of the organisation?

      When using Scrum, you'd expect to be able to plan at least a week worth of work in advance. Is that the case for your team? I would expect an ops team to have its share of unplanned incoming work during a week long sprint that was not known about before the sprint started, and that could make doing Scrum difficult.

      If you go the Kanban route, using the categories of work you give, you could define different 'swimming lanes' for each of those, measure cycle time for those, and work on decreasing that, and the variance in it.

      From a more organisational point of view, it might be interesting to take a look at where the different types of work are coming from, and seeing how you could be more closely working with the sources of the requests.
      Could you work more closely with the feature teams when supporting their work? Perhaps even have people work *in* those teams? Is the data import work related to new features? Can it be shared with the feature teams? Do other requests come directly from the business? Can you measure time spent on production issues, or system down-time, or reaction speed on problems, an so prioritise different monitoring and automation stories?

      Wouter


      On Wed, May 9, 2012 at 11:33 AM, danieleberge <daniele.berge@...> wrote:
       

      Hi guys,

      we are a component (backend hadoop) devops team that does the following work...
      - take care of a hadoop infrastructure operations (many of our products depend on that infrastructure)
      - further build the architecture of our system (our system was taken live by business, when it was still in research state, so we lack monitoring, etc. )
      - develop feature requests by the vertical product teams
      - do a lot of operations stuff, like importing data from many sources etc.

      We think that we are a strategic team within the organization as our company is heavily working with data.

      What we would like to do is somehow show the organization the value of our team and at the same time measure for ourselves, if we get better.

      We tried using Scrum as process and are sth. in between Scrum and Kanban right now (but that´s another story).

      With velocity we had the problem, that our development stories usually include some implementation combined with a lot of data importing and processing from different data-sources outside of our team´s control. So many times the implementation is easy to be done, but the work with data, and to actually produce value through the stories is a very long running task and often not predictible, since we depend on other teams and also many times have to debug those data-sources on the fly, since the domain knowledge is lacking. Commitment seems at times impossible to give.

      Anyway, we want to show the organization the value of our work and also see where we are as a team and learn, how we can get better.

      We are thinking about introducing a set of kpi´s for us in the dimensions of
      - operations
      - product development
      - value of the features we produce,...
      so a mix of business value, velocity and ops kpi´s.

      Has anyone experience with a team setup like this, and how to measure and market a team like that? Also, we are not sure if velocity is a good kpi for us. Is there maybe an alternative? For the business value kpi´s... since we do not develpp end-user features, how can we measure the value we produce?

      I am looking forward for an interesting discussion on devops teams, kpi´s and value! :)

      Cheers

      Daniel
      --- http://www.agileinberlin.com/ ---




      --
      Wouter Lagerweij         | wouter@...
    • danieleberge
      Hi Wouter, thanks for your extensive reply! Lots of ideas to chew on! The main reason for us wanting to show our value to the organisation is that historically
      Message 2 of 4 , May 11, 2012
      • 0 Attachment
        Hi Wouter,

        thanks for your extensive reply! Lots of ideas to chew on!

        The main reason for us wanting to show our value to the organisation is that historically our organisation, especially Product Management was frustrated with us, since we were not able to deliver all they think they need. From our perspective we just do not have enough bandwidth as a team and also our architecture isn´t mature enough to deliver quick wins for the feature teams. I hope that if we can make our work more transparent to the organisation this will give us a better standing and kpi´s also sth. we can use as a medium for negotiating with management to grant us more resources. Right now the value we deliver is not so much tangible.

        We are partly able to plan for the next sprint. Right now we try to cover all the operations stuff by putting one of our four people just on operations stuff per week. We are not in a stadium were we can predict so much the nature of the work to come. The last months we needed to integrate a new data source based on legacy databases. We will see in the coming weeks and learn more about the nature of our development. This comes mainly out of the fact, that our architecture comes out of research and is not plain development.

        Yep, the Kanban route could be a way we´ll have to experiment with.

        Your questions regarding working closer with the feature teams are good ones and I will have to chew on that. The team tried to distribute knowledge of our architecture to the feature teams, before I started working with the team. They said it didn´t work out that well since the architecture is too complex. The data input work can be part of new features, in fact right now often this is the case. Therefore we need access to datasources owned by either other data teams or IT and there are quite some political barriers to work effectively with those.

        The measuring questions are the one we have to tackle next.

        I think the main point I se in all this is that a dev + ops team is much more difficult to measure that a feature development team. There maybe sth. like velocity + defect rates is enough as kpi´s. But when having a dev team, doing ops and in development being highly dependent on other teams, the story gets much harder. Also I wonder, the whole question of how to set up the (horizontal) data teams is not one that is easy to answer. We are currently in the situation to work that stuff out.

        Thanks for your input.

        Daniel

        --- In scrumdevelopment@yahoogroups.com, Wouter Lagerweij <wouter@...> wrote:
        >
        > Hi Daniele,
        >
        > Interesting topic!
        >
        > What is the main reason your team feels you need to emphasize your team's
        > value to the rest of the organisation?
        >
        > When using Scrum, you'd expect to be able to plan at least a week worth of
        > work in advance. Is that the case for your team? I would expect an ops team
        > to have its share of unplanned incoming work during a week long sprint that
        > was not known about before the sprint started, and that could make doing
        > Scrum difficult.
        >
        > If you go the Kanban route, using the categories of work you give, you
        > could define different 'swimming lanes' for each of those, measure cycle
        > time for those, and work on decreasing that, and the variance in it.
        >
        > From a more organisational point of view, it might be interesting to take a
        > look at where the different types of work are coming from, and seeing how
        > you could be more closely working with the sources of the requests.
        > Could you work more closely with the feature teams when supporting their
        > work? Perhaps even have people work *in* those teams? Is the data import
        > work related to new features? Can it be shared with the feature teams? Do
        > other requests come directly from the business? Can you measure time spent
        > on production issues, or system down-time, or reaction speed on problems,
        > an so prioritise different monitoring and automation stories?
        >
        > Wouter
        >
        >
        > On Wed, May 9, 2012 at 11:33 AM, danieleberge <daniele.berge@...>wrote:
        >
        > > **
        > >
        > >
        > > Hi guys,
        > >
        > > we are a component (backend hadoop) devops team that does the following
        > > work...
        > > - take care of a hadoop infrastructure operations (many of our products
        > > depend on that infrastructure)
        > > - further build the architecture of our system (our system was taken live
        > > by business, when it was still in research state, so we lack monitoring,
        > > etc. )
        > > - develop feature requests by the vertical product teams
        > > - do a lot of operations stuff, like importing data from many sources etc.
        > >
        > > We think that we are a strategic team within the organization as our
        > > company is heavily working with data.
        > >
        > > What we would like to do is somehow show the organization the value of our
        > > team and at the same time measure for ourselves, if we get better.
        > >
        > > We tried using Scrum as process and are sth. in between Scrum and Kanban
        > > right now (but that´s another story).
        > >
        > > With velocity we had the problem, that our development stories usually
        > > include some implementation combined with a lot of data importing and
        > > processing from different data-sources outside of our team´s control. So
        > > many times the implementation is easy to be done, but the work with data,
        > > and to actually produce value through the stories is a very long running
        > > task and often not predictible, since we depend on other teams and also
        > > many times have to debug those data-sources on the fly, since the domain
        > > knowledge is lacking. Commitment seems at times impossible to give.
        > >
        > > Anyway, we want to show the organization the value of our work and also
        > > see where we are as a team and learn, how we can get better.
        > >
        > > We are thinking about introducing a set of kpi´s for us in the dimensions
        > > of
        > > - operations
        > > - product development
        > > - value of the features we produce,...
        > > so a mix of business value, velocity and ops kpi´s.
        > >
        > > Has anyone experience with a team setup like this, and how to measure and
        > > market a team like that? Also, we are not sure if velocity is a good kpi
        > > for us. Is there maybe an alternative? For the business value kpi´s...
        > > since we do not develpp end-user features, how can we measure the value we
        > > produce?
        > >
        > > I am looking forward for an interesting discussion on devops teams, kpi´s
        > > and value! :)
        > >
        > > Cheers
        > >
        > > Daniel
        > > --- http://www.agileinberlin.com/ ---
        > >
        > >
        > >
        >
        >
        >
        > --
        > Wouter Lagerweij | wouter@...
        > http://www.lagerweij.com | @wouterla <http://twitter.com/#!/wouterla>
        >
      • Wouter Lagerweij
        Hi Daniel, I can certainly understand the need to emphasize the value you re delivering as a team. It can be very difficult in a such a component team
        Message 3 of 4 , May 16, 2012
        • 0 Attachment
          Hi Daniel,

          I can certainly understand the need to emphasize the value you're delivering as a team. It can be very difficult in a such a component team situation!

          I've used the 'rotating operational issues guy' system in a Scrum team. It works fairly well, as long as the number of incoming issues stays within that bandwidth.

          As you say, a lot of things are just finding out how to get things going within your particular situation. I'd only want to add that, from the agile point of view, focus on making issues visible, and on shortening the feedback cycles.
          So make it very visible when you're waiting on external parties. As 'impediments', maybe adding a red post-it on the scrum board, but also by measuring when things get stuck, and for how long, to get you some numbers. In the end, those types of issues are for management to pick up, but they'll need data.

          good luck, and let us know the results of your experiments!

          Wouter

          On Fri, May 11, 2012 at 7:15 PM, danieleberge <daniele.berge@...> wrote:
           

          Hi Wouter,

          thanks for your extensive reply! Lots of ideas to chew on!

          The main reason for us wanting to show our value to the organisation is that historically our organisation, especially Product Management was frustrated with us, since we were not able to deliver all they think they need. From our perspective we just do not have enough bandwidth as a team and also our architecture isn´t mature enough to deliver quick wins for the feature teams. I hope that if we can make our work more transparent to the organisation this will give us a better standing and kpi´s also sth. we can use as a medium for negotiating with management to grant us more resources. Right now the value we deliver is not so much tangible.

          We are partly able to plan for the next sprint. Right now we try to cover all the operations stuff by putting one of our four people just on operations stuff per week. We are not in a stadium were we can predict so much the nature of the work to come. The last months we needed to integrate a new data source based on legacy databases. We will see in the coming weeks and learn more about the nature of our development. This comes mainly out of the fact, that our architecture comes out of research and is not plain development.

          Yep, the Kanban route could be a way we´ll have to experiment with.

          Your questions regarding working closer with the feature teams are good ones and I will have to chew on that. The team tried to distribute knowledge of our architecture to the feature teams, before I started working with the team. They said it didn´t work out that well since the architecture is too complex. The data input work can be part of new features, in fact right now often this is the case. Therefore we need access to datasources owned by either other data teams or IT and there are quite some political barriers to work effectively with those.

          The measuring questions are the one we have to tackle next.

          I think the main point I se in all this is that a dev + ops team is much more difficult to measure that a feature development team. There maybe sth. like velocity + defect rates is enough as kpi´s. But when having a dev team, doing ops and in development being highly dependent on other teams, the story gets much harder. Also I wonder, the whole question of how to set up the (horizontal) data teams is not one that is easy to answer. We are currently in the situation to work that stuff out.

          Thanks for your input.

          Daniel



          --- In scrumdevelopment@yahoogroups.com, Wouter Lagerweij <wouter@...> wrote:
          >
          > Hi Daniele,
          >
          > Interesting topic!
          >
          > What is the main reason your team feels you need to emphasize your team's
          > value to the rest of the organisation?
          >
          > When using Scrum, you'd expect to be able to plan at least a week worth of
          > work in advance. Is that the case for your team? I would expect an ops team
          > to have its share of unplanned incoming work during a week long sprint that
          > was not known about before the sprint started, and that could make doing
          > Scrum difficult.
          >
          > If you go the Kanban route, using the categories of work you give, you
          > could define different 'swimming lanes' for each of those, measure cycle
          > time for those, and work on decreasing that, and the variance in it.
          >
          > From a more organisational point of view, it might be interesting to take a
          > look at where the different types of work are coming from, and seeing how
          > you could be more closely working with the sources of the requests.
          > Could you work more closely with the feature teams when supporting their
          > work? Perhaps even have people work *in* those teams? Is the data import
          > work related to new features? Can it be shared with the feature teams? Do
          > other requests come directly from the business? Can you measure time spent
          > on production issues, or system down-time, or reaction speed on problems,
          > an so prioritise different monitoring and automation stories?
          >
          > Wouter
          >
          >
          > On Wed, May 9, 2012 at 11:33 AM, danieleberge <daniele.berge@...>wrote:
          >
          > > **

          > >
          > >
          > > Hi guys,
          > >
          > > we are a component (backend hadoop) devops team that does the following
          > > work...
          > > - take care of a hadoop infrastructure operations (many of our products
          > > depend on that infrastructure)
          > > - further build the architecture of our system (our system was taken live
          > > by business, when it was still in research state, so we lack monitoring,
          > > etc. )
          > > - develop feature requests by the vertical product teams
          > > - do a lot of operations stuff, like importing data from many sources etc.
          > >
          > > We think that we are a strategic team within the organization as our
          > > company is heavily working with data.
          > >
          > > What we would like to do is somehow show the organization the value of our
          > > team and at the same time measure for ourselves, if we get better.
          > >
          > > We tried using Scrum as process and are sth. in between Scrum and Kanban
          > > right now (but that´s another story).
          > >
          > > With velocity we had the problem, that our development stories usually
          > > include some implementation combined with a lot of data importing and
          > > processing from different data-sources outside of our team´s control. So
          > > many times the implementation is easy to be done, but the work with data,
          > > and to actually produce value through the stories is a very long running
          > > task and often not predictible, since we depend on other teams and also
          > > many times have to debug those data-sources on the fly, since the domain
          > > knowledge is lacking. Commitment seems at times impossible to give.
          > >
          > > Anyway, we want to show the organization the value of our work and also
          > > see where we are as a team and learn, how we can get better.
          > >
          > > We are thinking about introducing a set of kpi´s for us in the dimensions
          > > of
          > > - operations
          > > - product development
          > > - value of the features we produce,...
          > > so a mix of business value, velocity and ops kpi´s.
          > >
          > > Has anyone experience with a team setup like this, and how to measure and
          > > market a team like that? Also, we are not sure if velocity is a good kpi
          > > for us. Is there maybe an alternative? For the business value kpi´s...
          > > since we do not develpp end-user features, how can we measure the value we
          > > produce?
          > >
          > > I am looking forward for an interesting discussion on devops teams, kpi´s
          > > and value! :)
          > >
          > > Cheers
          > >
          > > Daniel
          > > --- http://www.agileinberlin.com/ ---
          > >
          > >
          > >
          >
          >
          >
          > --
          > Wouter Lagerweij | wouter@...
          > http://www.lagerweij.com | @wouterla <http://twitter.com/#!/wouterla>
          >




          --
          Wouter Lagerweij         | wouter@...
        Your message has been successfully submitted and would be delivered to recipients shortly.