Loading ...
Sorry, an error occurred while loading the content.
 

Philosophical question: is this about FOSS? or Agile?

Expand Messages
  • Esther Schindler
    I wrote a blog post about apparently contradictory research results from ... analyst Carey Schwaber, the average time to resolve an enterprise application
    Message 1 of 6 , Sep 29, 2007
      I wrote a blog post about apparently contradictory research results from
      BMC & Forrester and from Evans Data. The key point is this:

      >>According to a survey commissioned by BMC and conducted by Forrester
      analyst Carey Schwaber, the average time to resolve an enterprise
      application problem is 6.9 days. Ten percent of those problems take 10
      days to solve. Yet, for open-source developers, the time reported
      between bug reports and the community's response is far, far less.>>

      http://advice.cio.com/esther_schindler/enterprise_developers_programming_speed_check_time_to_fix_bugs_not_so_much


      But, based on a couple of happy comments from people in the open source
      community, I now wonder if it actually has anything to do with FOSS but
      more to do with the (not necessarily named) methodology used on a lot of
      open source projects. That is... is this more of an Agile win than
      something to do with the difference between enterprise culture and the
      FOSS way of life?

      Esther
    • Jon Eaves
      ... I work for large Australian bank (www.anz.com) and a reasonably successful and widely used open source project (www.bouncycastle.org). We use Agile
      Message 2 of 6 , Sep 29, 2007
        Esther Schindler wrote:
        > I wrote a blog post about apparently contradictory research results from
        > BMC & Forrester and from Evans Data. The key point is this:
        >
        > >>According to a survey commissioned by BMC and conducted by Forrester
        > analyst Carey Schwaber, the average time to resolve an enterprise
        > application problem is 6.9 days. Ten percent of those problems take 10
        > days to solve. Yet, for open-source developers, the time reported
        > between bug reports and the community's response is far, far less.>>
        >
        > http://advice.cio.com/esther_schindler/enterprise_developers_programming_speed_check_time_to_fix_bugs_not_so_much
        >
        >
        > But, based on a couple of happy comments from people in the open source
        > community, I now wonder if it actually has anything to do with FOSS but
        > more to do with the (not necessarily named) methodology used on a lot of
        > open source projects. That is... is this more of an Agile win than
        > something to do with the difference between enterprise culture and the
        > FOSS way of life?

        I work for large Australian bank (www.anz.com) and a reasonably successful and widely
        used open source project (www.bouncycastle.org).

        We use Agile internally at the bank for our projects (yes really, and we're getting good
        mileage from it) and for Bouncycastle it's far more of a code and fix. Now let me clear
        up on that, it's not the bad sort of code and fix, it's the "new crypto algorithm has been
        ratified, who's going to do it?". We have large regression test suites in both cases, lots
        of unit tests and pretty good customers ;-)

        In ANZ we can get a code fix in roughly the same time as Bouncycastle. Anywhere between 2
        and 4 hours is a "normal" fix time for a well bounded defect. Anywhere around 24 to 48 hours
        for an "interface defect". These are when external systems don't operate as their specifications
        claim. We have a large number of legacy systems in ANZ, and there is a large number of
        interoperability areas within the crypto world (think of the large numbers of versions of PGP)

        The difference between the two is culture _external_ to the project team.

        Once a code fix has been done within our ANZ software team, then the "corporate machine" takes
        over and there is anywhere between 3 and 5 different teams required to "sign off" on the
        changes before it goes live. It then is pushed live by a team completely unrelated to the
        software team, sometimes with errors which require re-work/re-fixing.

        Within Bouncycastle, the Java team (of which there is 2) send an email saying "I've done this,
        can you make sure it works for you?" (there are some environmental complexities within the
        number of JVM's we support, from JavaME -> Java5 using the same codebase). The library is
        then dropped into a beta area, our customers are notified on our mailing list (including a
        direct email to the defect reporter).

        It stays in beta form until our next release (normally 2-3 months between major releases).

        At ANZ there is a fear driven, high ceremony approach to releases. At Bouncycastle there is
        a change embracing, low ceremony approach to releases. Both achieve roughly the same outcome
        with regression failures, and/or incomplete fixes.

        So, in my experience, it's normally a result of the culture _external_ to the team that has a
        greater impact than the internal workings of the project team, holding most other things
        equal. Not surprising to most people, and what you've concluded at the end of your article,
        but probably something not well understood.

        Cheers,
        -- jon

        --
        Jon Eaves <jon@...>
        http://www.eaves.org/blog/
        Co-Author of "Apache Tomcat Bible", "Professional Tomcat 5", "Beginning JavaServer Pages"
      • Charlie Poole
        Hi Esther, I got myself into open source in the first place in order to explore the similarities and differences with agile - specifically extreme programming.
        Message 3 of 6 , Sep 29, 2007
          Hi Esther,

          I got myself into open source in the first place in order to
          explore the similarities and differences with agile - specifically
          extreme programming. My conclusion after a few years of it is that
          there are similarities and there are differences... :-)

          Open source was talking about (lower-case) agile concepts long
          before there was an agile movement. I'm pretty sure that the
          open source movement was a big influence on the early leaders
          of our movement - although you'd have to ask them to be sure.

          So part of the advantage of FOSS projects comes - I believe - from
          their "agility." We have to be careful how we say that though,
          since many open source folks are pretty adamant that they didn't
          learn their "agility" from us. They correctly state they were
          doing (some of) our agile practices long before we ever heard
          of them.

          Another reason for the advantage - one you haven't mentionned - is
          the commonn acceptance that bugs (or "issues") are a normal part of
          life. It's part of the culture of FOSS - at least the parts of it
          I have seen - to respect people who own up to their mistakes and
          fix them quickly. Unfortunately, that's not a model followed in
          a lot of enterprise situations.

          Finally, open source has a serious advantage at the deployment
          end of things. Once a bug is fixed, it's immediately available
          for anyone who is willing to build from source. And for those
          who don't, many projects release /nightly/ binary packages. The
          only practical way to achieve this in a commercial shrinkwrap
          setting is through a subscription model.

          Bear in mind, however, that I'm no expert. I've only seen a part
          of the elephant, but enough to know it's a pretty big beast.

          Charlie

          > -----Original Message-----
          > From: extremeprogramming@yahoogroups.com
          > [mailto:extremeprogramming@yahoogroups.com] On Behalf Of
          > Esther Schindler
          > Sent: Saturday, September 29, 2007 1:34 PM
          > To: extremeprogramming@yahoogroups.com
          > Subject: [XP] Philosophical question: is this about FOSS? or Agile?
          >
          > I wrote a blog post about apparently contradictory research
          > results from BMC & Forrester and from Evans Data. The key
          > point is this:
          >
          > >>According to a survey commissioned by BMC and conducted by
          > Forrester analyst Carey Schwaber, the average time to resolve
          > an enterprise application problem is 6.9 days. Ten percent of
          > those problems take 10 days to solve. Yet, for open-source
          > developers, the time reported between bug reports and the
          > community's response is far, far less.>>
          >
          > http://advice.cio.com/esther_schindler/enterprise_developers_p
          > rogramming_speed_check_time_to_fix_bugs_not_so_much
          >
          >
          > But, based on a couple of happy comments from people in the
          > open source community, I now wonder if it actually has
          > anything to do with FOSS but more to do with the (not
          > necessarily named) methodology used on a lot of open source
          > projects. That is... is this more of an Agile win than
          > something to do with the difference between enterprise
          > culture and the FOSS way of life?
          >
          > Esther
          >
          >
          > To Post a message, send it to: extremeprogramming@...
          >
          > To Unsubscribe, send a blank message to:
          > extremeprogramming-unsubscribe@...
          >
          > ad-free courtesy of objectmentor.com
          > Yahoo! Groups Links
          >
          >
          >
          >
        • julescov
          I think a lot of the difference comes from the corporate environment, particularly with regard to pre-release testing. I was, not too long ago, involved in a
          Message 4 of 6 , Sep 30, 2007
            I think a lot of the difference comes from the corporate environment,
            particularly with regard to pre-release testing.

            I was, not too long ago, involved in a rather non-agile project. My
            company has always had somewhat agile tendencies (although we only
            recently realised that's what they were), so it was somewhat of a
            culture shock when one of our clients sold his business to a large
            multinational with a rather rigid waterfall-style process and we
            agreed to modify the software we'd produced to fit into their existing
            systems.

            My guess is that the process they used there is rather similar to the
            way a lot of enterprise software is developed: a small team of
            programmers (in this case I joined an existing team of 2 programmers
            and one project manager who didn't do any programming work) working on
            a product with dependencies on other projects (in this case handled by
            a team on subcontract from another company). The local team had no
            authority to make changes to database setup (which had to be handled
            by the on-site DBA for the test system and an off-site DBA for the
            live system), to the deployed web site we were working on (handled by
            another subcontracter) or even to approve releases (handled by a
            testing team on an entirely different continent).

            While I was working on that project, we had a turn-around time of 4
            days just to make a release in response to a bug report. The process
            was something like this:

            Day 1:
            * end user reports problem to customer services
            * customer services sends an e-mail to the testing team, which then
            has to wait for the testing team's time zone difference to make it
            office hours there
            * testing team (who don't really understand how the product works) try
            to verify the problem. (If they fail, they send a message to the
            development team asking for suggestions how to verify it. This
            usually adds an extra day to the process, because there's only a one
            hour window where the two teams can communicate in real time)

            Day 2:
            * bug report is received by project manager, who tries to guess which
            of the 4 layers (user interface, front end server, back end business
            logic layer, service layer for integration with existing system)
            contains the bug. She guesses this with only slightly better than 50%
            accuracy. Occasions where she guesses incorrectly add an extra half
            day - day to the time to fix.
            * because I'm working part time and on contract, if the bug is
            assigned to me there's a good chance I'm not already on site. Due to
            security issues, I'm not allowed sufficient access from off-site to be
            able to work effectively. Most of the time, therfore, I come in the
            next day.

            Day 3:
            * Work on fixing the bug begins.

            Day 3 + n (where n is number of complete days required to fix):
            * Work on fixing the bug ends & project manager is informed that the
            fix is ready for testing.
            * Testing begins shortly before close of work (assuming the testing
            team are available and haven't been booked by somebody else). It is
            almost certainly not finished by the end of the working day.

            Day 4 + n:
            * Test results come back, assuming success, external contractor who
            maintains public-facing web site is contacted and asked to authorize
            new version upload.

            Day 5 + n:
            * New version goes live in early morning.

            There is, therefore, a 5 day minimum turnaround on a bug due to the
            beaurocracy required to handle it. In most cases 6-7 was more
            realistic. This doesn't include the additional overhead where the fix
            required cooperation between two different layers (which reaches its
            peak when one of those layers is the layer I worked on and one of them
            is the layer provided by the other subcontractor).

            This compares to the typical open source project, where the person
            receiving the bug report is a developer (who understands the project
            in substantially more depth than the customer service team, testing
            team and project manager who have to handle the bug in the corporate
            scenario before it gets to a developer) who more often than not is
            able to validate the bug himself (no test team required), fix the code
            directly (no code ownership issues), test the fix himself (no test
            team required), check the code in and perform a release himself (no
            external subcontracters required).

            Larger open source projects (I'm thinking primarily of Mozilla here)
            tend to be more corporation-like in structure, with bug reports
            initially handled by people who don't actually know the code very
            well, a peer review process for fixes (admittedly not as bad as the
            outsourced-to-India test contractor), etc. OTOH, some other open
            source projects handle growth in size much better (I've had good
            experiences reporting bugs to the Eclipse project, for instance, and
            working with OpenOffice.org hasn't been an entirely negative
            experience either). Agility may or may not have anything to do with
            it, but I do note that I have had better experiences with projects
            that are generally noted for their agile approach.
          • William Pietri
            ... It s both. Open-source projects are much more agile than your average closed-source project, certainly. (As Charlie points out, however, they didn t get
            Message 5 of 6 , Sep 30, 2007
              Esther Schindler wrote:
              > But, based on a couple of happy comments from people in the open source
              > community, I now wonder if it actually has anything to do with FOSS but
              > more to do with the (not necessarily named) methodology used on a lot of
              > open source projects. That is... is this more of an Agile win than
              > something to do with the difference between enterprise culture and the
              > FOSS way of life?
              >


              It's both.

              Open-source projects are much more agile than your average closed-source
              project, certainly. (As Charlie points out, however, they didn't get
              that from our community. And I don't think we got much of that from
              them, although I'd love to hear what Kent and Ron say.) That agility
              certainly leads to faster resolution.

              But it's also a question of the power structures involved. Most
              corporations have a top-down approach to things. They also have a very
              limited number of people allowed to work on (and often, even view) a
              particular code base. On an open-source project, though, it's a more
              bottom-up culture. Anybody can look at the code and suggest a fix, and
              the cultural expectation is that good patches should be applied quickly.

              Even if the trunk maintainers don't accept the patch you need
              immediately, you can always build your own copy. For a great example of
              that at work, you can look at the history of the Apache server, which
              started life as a set of patches against an unmaintained code base.[1]
              Thus it's name as "a patch-y" server.

              One interesting consequence of this is the difference in the code bases.
              From what I've seen, the code on successful open source projects is on
              average much more readable, much more accessible than your average
              corporate project. And for your purposes, Esther, I think that makes a
              big difference. If working with the code is easy, it's much more likely
              that some non-central player will jump in and fix an interesting bug or
              add a neat little feature.

              William



              [1] http://en.wikipedia.org/wiki/Apache_Web_Server
            • Esther Schindler
              ... Charlie and all... What cool responses! I feel very much smarter now. I encourage you to post replies to the blog entry
              Message 6 of 6 , Oct 1, 2007
                On Sep 29, 2007, at 3:33 PM, Charlie Poole wrote:
                > I got myself into open source in the first place in order to
                > explore the similarities and differences with agile - specifically
                > extreme programming. My conclusion after a few years of it is that
                > there are similarities and there are differences... :-)

                Charlie and all...

                What cool responses! I feel very much smarter now. <gazing admiringly
                at you all>

                I encourage you to post replies to the blog entry (too). It's nice to
                have Eric Raymond respond to it, but we don't want him thinking that
                it's *all* about open source, do we?

                Esther
              Your message has been successfully submitted and would be delivered to recipients shortly.