Loading ...
Sorry, an error occurred while loading the content.

Integrate or Stovepipe?

Expand Messages
  • Dave Rooney
    I know that the conventional XP wisdom regarding frameworks is to build at least two applications and refactor the commonalities out into a framework to
    Message 1 of 9 , Nov 2, 2005
    • 0 Attachment
      I know that the conventional XP wisdom regarding frameworks is to build
      at least two applications and refactor the commonalities out into a
      framework to eliminate the duplication.
      However, what does the group think about dealing with the integration of
      multiple applications?

      At a current client, we have two different groups using two different
      approaches to delivering applications. The group with whom I currently
      work has a policy where any new functionality to support the business is
      integrated into the existing system. This results in a common code base
      for what can appear to the end user to be multiple applications. That's
      a "Good Thing". However, it does take a good chunk of work to properly
      integrate new functionality in terms of figuring out where to put
      things, and refactoring the code appropriately. Over the 2.5 year time
      period that this system has existed, it has taken gradually longer to
      integrate each new piece of functionality since the system continues to
      grow and the refactoring efforts take longer. I have mentioned before
      that there is a substantial technical debt in this system, and this
      continues to be an issue. So, the end result is that our releases are
      taking longer over time for an equivalent amount of functionality.

      Meanwhile, there is another team that uses a different approach. Their
      philosophy is to build small, simple applications to meet specific
      business criteria, and create interfaces between the applications as
      required. What I'm seeing from them is that they're able to react to
      business requests faster than we can, although they have a considerable
      amount of code duplicated between applications. While that team could be
      considered 'agile', they aren't specifically using an agile development
      method.

      So the question is, which approach is better? There are definitely
      merits to a single, integrated system, but in practice we've found that
      it certainly isn't easy to do and that our ability to deliver business
      value has diminished over time (based on our team's track record - we're
      still way ahead of most other teams). There are also merits to building
      small stovepipe applications with interfaces, but you then have to deal
      with duplicated code, possibly in multiple incompatible versions.

      I'm wondering if a hybrid approach would be better - build the stovepipe
      application to meet the immediate business need, build any required
      interfaces, but assume that over time that stovepipe will be integrated
      into the system to eliminate duplication.

      Thoughts?

      Dave Rooney
      Mayford Technologies
      http://www.mayford.ca
    • Greg Akins
      I m in almost the exact same situation. I think that with care, the integrated approach is better, but I understand how if the technical debt accumulates that
      Message 2 of 9 , Nov 2, 2005
      • 0 Attachment
        I'm in almost the exact same situation.

        I think that with care, the integrated approach is
        better, but I understand how if the technical debt
        accumulates that it can become difficult to refactor.

        However, what's the cost of doing it the other way.
        As that team duplicates code, what's the cost to the
        business.

        In my organization, we have enough turnover that
        applications that were developed less than a year ago,
        are thrown away and redeveloped simply because the
        current staff aren't really aware of them.

        They can turn over requests quicker, but in the end
        they're creating less value with more people.

        --- Dave Rooney <dave.rooney@...> wrote:

        > I know that the conventional XP wisdom regarding
        > frameworks is to build
        > at least two applications and refactor the
        > commonalities out into a
        > framework to eliminate the duplication.
        > However, what does the group think about dealing
        > with the integration of
        > multiple applications?
        >
        > At a current client, we have two different groups
        > using two different
        > approaches to delivering applications. The group
        > with whom I currently
        > work has a policy where any new functionality to
        > support the business is
        > integrated into the existing system. This results
        > in a common code base
        > for what can appear to the end user to be multiple
        > applications. That's
        > a "Good Thing". However, it does take a good chunk
        > of work to properly
        > integrate new functionality in terms of figuring out
        > where to put
        > things, and refactoring the code appropriately.
        > Over the 2.5 year time
        > period that this system has existed, it has taken
        > gradually longer to
        > integrate each new piece of functionality since the
        > system continues to
        > grow and the refactoring efforts take longer. I
        > have mentioned before
        > that there is a substantial technical debt in this
        > system, and this
        > continues to be an issue. So, the end result is
        > that our releases are
        > taking longer over time for an equivalent amount of
        > functionality.
        >
        > Meanwhile, there is another team that uses a
        > different approach. Their
        > philosophy is to build small, simple applications to
        > meet specific
        > business criteria, and create interfaces between the
        > applications as
        > required. What I'm seeing from them is that they're
        > able to react to
        > business requests faster than we can, although they
        > have a considerable
        > amount of code duplicated between applications.
        > While that team could be
        > considered 'agile', they aren't specifically using
        > an agile development
        > method.
        >
        > So the question is, which approach is better? There
        > are definitely
        > merits to a single, integrated system, but in
        > practice we've found that
        > it certainly isn't easy to do and that our ability
        > to deliver business
        > value has diminished over time (based on our team's
        > track record - we're
        > still way ahead of most other teams). There are
        > also merits to building
        > small stovepipe applications with interfaces, but
        > you then have to deal
        > with duplicated code, possibly in multiple
        > incompatible versions.
        >
        > I'm wondering if a hybrid approach would be better -
        > build the stovepipe
        > application to meet the immediate business need,
        > build any required
        > interfaces, but assume that over time that stovepipe
        > will be integrated
        > into the system to eliminate duplication.
        >
        > Thoughts?
        >
        > Dave Rooney
        > Mayford Technologies
        > http://www.mayford.ca
        >
        >
        >
        >
        > To Post a message, send it to:
        > extremeprogramming@...
        >
        > To Unsubscribe, send a blank message to:
        > extremeprogramming-unsubscribe@...
        >
        > ad-free courtesy of objectmentor.com
        > Yahoo! Groups Links
        >
        > http://groups.yahoo.com/group/extremeprogramming/
        >
        > extremeprogramming-unsubscribe@yahoogroups.com
        >
        >
        >
        >
        >
        >
      • Tim Haughton
        ... Hi Dave, an extremely good question. In my experience, generic application frameworks don t emerge, at least not in a useful cost efficient way. This seems
        Message 3 of 9 , Nov 2, 2005
        • 0 Attachment
          On 02/11/05, Dave Rooney <dave.rooney@...> wrote:
          > I know that the conventional XP wisdom regarding frameworks is to build
          > at least two applications and refactor the commonalities out into a
          > framework to eliminate the duplication.
          > However, what does the group think about dealing with the integration of
          > multiple applications?

          Hi Dave, an extremely good question.

          In my experience, generic application frameworks don't emerge, at
          least not in a useful cost efficient way. This seems to be backed up
          by your observations. Although if you tackled the technical debt in
          your system, you might observe something different.

          There are also logistical (and often political) problems to overcome.
          For example, the product owner of the second and third products don't
          often want to pay for the refactorings of the first product.

          The approach I've seen work the best is having a common set of
          components/services rather than a generic application framework.
          Frameworks work best (IMHO) when developed *as* a generic framework.

          At the end of the day, you can't argue with your empirical evidence.
          If producing a series of small disparate applications provides your
          customer with better value, what possible justification could you
          offer to not prefer that technique?

          It would be really neat to see a third team using a bought in generic
          application framework to compare that against your existing efforts.

          Cheers,

          Tim Haughton
        • Ron Jeffries
          ... Dave ... I missed your description of the tangible benefits of the single integrated approach. Could you review those, and the benefits of the stovepipe
          Message 4 of 9 , Nov 2, 2005
          • 0 Attachment
            On Wednesday, November 2, 2005, at 5:16:32 AM, Dave Rooney wrote:

            > So the question is, which approach is better? There are definitely
            > merits to a single, integrated system, but in practice we've found that
            > it certainly isn't easy to do and that our ability to deliver business
            > value has diminished over time (based on our team's track record - we're
            > still way ahead of most other teams). There are also merits to building
            > small stovepipe applications with interfaces, but you then have to deal
            > with duplicated code, possibly in multiple incompatible versions.

            Dave ... I missed your description of the tangible benefits of the
            single integrated approach. Could you review those, and the benefits
            of the "stovepipe" approach? Maybe make a little table of benefits
            and drawbacks vs integrated and stovepipe?

            Ron Jeffries
            www.XProgramming.com
            Agility might be said to be about encountering
            all the problems so early and so often that the
            effort to fix them is less than the pain of enduring them.
          • yahoogroups@jhrothjr.com
            below. ... From: Dave Rooney To: extremeprogramming@yahoogroups.com
            Message 5 of 9 , Nov 2, 2005
            • 0 Attachment
              below.


              ----- Original Message -----
              From: "Dave Rooney" <dave.rooney.at.mayford.ca@...>
              To: "extremeprogramming@yahoogroups.com"
              <extremeprogramming.at.yahoogroups.com@...>
              Sent: Wednesday, November 02, 2005 6:16 AM
              Subject: [XP] Integrate or Stovepipe?


              >I know that the conventional XP wisdom regarding frameworks is to build
              > at least two applications and refactor the commonalities out into a
              > framework to eliminate the duplication.
              > However, what does the group think about dealing with the integration of
              > multiple applications?
              >
              > At a current client, we have two different groups using two different
              > approaches to delivering applications. The group with whom I currently
              > work has a policy where any new functionality to support the business is
              > integrated into the existing system. This results in a common code base
              > for what can appear to the end user to be multiple applications. That's
              > a "Good Thing". However, it does take a good chunk of work to properly
              > integrate new functionality in terms of figuring out where to put
              > things, and refactoring the code appropriately. Over the 2.5 year time
              > period that this system has existed, it has taken gradually longer to
              > integrate each new piece of functionality since the system continues to
              > grow and the refactoring efforts take longer. I have mentioned before
              > that there is a substantial technical debt in this system, and this
              > continues to be an issue. So, the end result is that our releases are
              > taking longer over time for an equivalent amount of functionality.
              >
              > Meanwhile, there is another team that uses a different approach. Their
              > philosophy is to build small, simple applications to meet specific
              > business criteria, and create interfaces between the applications as
              > required. What I'm seeing from them is that they're able to react to
              > business requests faster than we can, although they have a considerable
              > amount of code duplicated between applications. While that team could be
              > considered 'agile', they aren't specifically using an agile development
              > method.
              >
              > So the question is, which approach is better? There are definitely
              > merits to a single, integrated system, but in practice we've found that
              > it certainly isn't easy to do and that our ability to deliver business
              > value has diminished over time (based on our team's track record - we're
              > still way ahead of most other teams). There are also merits to building
              > small stovepipe applications with interfaces, but you then have to deal
              > with duplicated code, possibly in multiple incompatible versions.
              >
              > I'm wondering if a hybrid approach would be better - build the stovepipe
              > application to meet the immediate business need, build any required
              > interfaces, but assume that over time that stovepipe will be integrated
              > into the system to eliminate duplication.
              >
              > Thoughts?

              What seems to be missing in the first case is a phrase that's
              much denegrated around here: Architecture. If new functionality
              is taking longer to do, rather than the same or even less time, you've
              got an incipient Big Ball of Mud.

              The code debt is probably obscuring the architecture that
              wants to emerge.

              In the second case, I suspect that there is a framework,
              or possibly several frameworks, just begging to emerge
              from the code.

              I wouldn't say, at this point, that one approach
              is better than the other in all cases.

              John Roth


              >
              > Dave Rooney
              > Mayford Technologies
              > http://www.mayford.ca
              >
              >
              >
              >
              > To Post a message, send it to: extremeprogramming@...
              >
              > To Unsubscribe, send a blank message to:
              > extremeprogramming-unsubscribe@...
              >
              > ad-free courtesy of objectmentor.com
              > Yahoo! Groups Links
              >
              >
              >
              >
              >
              >
              >
              >
            • Jessamyn Smith
              ... I am wondering why it is that adding each new piece of functionality takes longer than the one before. I understand that the system as a whole is growing,
              Message 6 of 9 , Nov 2, 2005
              • 0 Attachment
                On 11/2/05, Dave Rooney <dave.rooney@...> wrote:
                >
                > I know that the conventional XP wisdom regarding frameworks is to build
                > at least two applications and refactor the commonalities out into a
                > framework to eliminate the duplication.
                > However, what does the group think about dealing with the integration of
                > multiple applications?
                >
                > At a current client, we have two different groups using two different
                > approaches to delivering applications. The group with whom I currently
                > work has a policy where any new functionality to support the business is
                > integrated into the existing system. This results in a common code base
                > for what can appear to the end user to be multiple applications. That's
                > a "Good Thing". However, it does take a good chunk of work to properly
                > integrate new functionality in terms of figuring out where to put
                > things, and refactoring the code appropriately. Over the 2.5 year time
                > period that this system has existed, it has taken gradually longer to
                > integrate each new piece of functionality since the system continues to
                > grow and the refactoring efforts take longer. I have mentioned before
                > that there is a substantial technical debt in this system, and this
                > continues to be an issue. So, the end result is that our releases are
                > taking longer over time for an equivalent amount of functionality.

                I am wondering why it is that adding each new piece of functionality takes
                longer than the one before. I understand that the system as a whole is
                growing, but why are you having to refactor ever larger parts of the system
                with each addition? That implies to me that the new additions are not well
                designed, or perhaps the new applications are entirely incompatible with
                existing functionality.
                I was in a (perhaps) similar situation at my last job. Initially we had a
                large number of separate applications, with a great deal of duplicated code.
                We were continually pulling common functionality out into libraries,
                simplifying existing applications. Despite adding a great deal of new
                functionality, our codebase was not growing much overall, because of the
                ongoing refactoring efforts. Adding new functionality got steadily easier
                and safer, thanks to unit tests, existing functions, etc. Admittedly my
                first attempts at pulling out duplicated code weren't very elegant, and the
                resulting functions often required significant tweaking when the next new
                feature came along. As I grew more experienced, I made better refactorings
                and this issue largely disappeared.
                Perhaps I am not correctly picturing your system, but I just don't see why
                every new addition should require refactoring everything that existed
                previously. At least some of the time, shouldn't previous refactorings have
                resulted in functions that are already usable?
                Perhaps the crucial difference here is that we did not have one monolithic
                application with multiple interfaces; rather we had multiple applications
                using libraries. Over time, the libraries got much bigger and the
                applications themselves got much smaller, even with all the new features.
                Jessamyn


                [Non-text portions of this message have been removed]
              • Dave Rooney
                ... I m thinking as I type here... the table may come later. The main benefit of the integrated application is a single code base. Ideally, that code base
                Message 7 of 9 , Nov 2, 2005
                • 0 Attachment
                  Ron Jeffries wrote:

                  >On Wednesday, November 2, 2005, at 5:16:32 AM, Dave Rooney wrote:
                  >
                  >
                  >
                  >>So the question is, which approach is better? There are definitely
                  >>merits to a single, integrated system, but in practice we've found that
                  >>it certainly isn't easy to do and that our ability to deliver business
                  >>value has diminished over time (based on our team's track record - we're
                  >>still way ahead of most other teams). There are also merits to building
                  >>small stovepipe applications with interfaces, but you then have to deal
                  >>with duplicated code, possibly in multiple incompatible versions.
                  >>
                  >>
                  >
                  >Dave ... I missed your description of the tangible benefits of the
                  >single integrated approach. Could you review those, and the benefits
                  >of the "stovepipe" approach? Maybe make a little table of benefits
                  >and drawbacks vs integrated and stovepipe?
                  >
                  >
                  I'm thinking as I type here... the table may come later.

                  The main benefit of the integrated application is a single code base.
                  Ideally, that code base would be well factored, but ours has some
                  significant warts. One of the requirements of these integrated
                  applications is that there is a single place where 'client' information
                  is being stored. At some point, although not yet, analysts want to be
                  able to "search on anything about a client, and get everything back",
                  regardless of the source. So, we're using the same code for reading,
                  writing, and shuttling client information around for 5 different lines
                  of business. That's the good part.

                  The bad part is that business functions such as validation aren't done
                  very well. We're in the process of finally factoring it out, but
                  there's a long way to go there before I'd say it's production-ready.
                  Another bad part is that there is a lot of code that is only used in one
                  context, and is completely unnecessary in others. To some on the team,
                  this provides flexibility and simplicity. I feel the opposite - code
                  that isn't used is anything but simple to me, and you have to write more
                  code to properly handle nulls, etc. In short, it smells.

                  I'll respond more to Jessamyn Smith's message - some of the thoughts
                  there resonate with me.

                  Dave Rooney
                  Mayford Technologies
                  http://www.mayford.ca
                • Ron Jeffries
                  ... If adding each new piece of functionality takes longer than the one before, there is one and only one likely reason: we are making the design worse with
                  Message 8 of 9 , Nov 2, 2005
                  • 0 Attachment
                    Around Wednesday, November 2, 2005, 2:54:41 PM, Jessamyn Smith wrote:

                    > On 11/2/05, Dave Rooney <dave.rooney@...> wrote:
                    >>
                    >> I know that the conventional XP wisdom regarding frameworks is to build
                    >> at least two applications and refactor the commonalities out into a
                    >> framework to eliminate the duplication.
                    >> However, what does the group think about dealing with the integration of
                    >> multiple applications?
                    >>
                    >> At a current client, we have two different groups using two different
                    >> approaches to delivering applications. The group with whom I currently
                    >> work has a policy where any new functionality to support the business is
                    >> integrated into the existing system. This results in a common code base
                    >> for what can appear to the end user to be multiple applications. That's
                    >> a "Good Thing". However, it does take a good chunk of work to properly
                    >> integrate new functionality in terms of figuring out where to put
                    >> things, and refactoring the code appropriately. Over the 2.5 year time
                    >> period that this system has existed, it has taken gradually longer to
                    >> integrate each new piece of functionality since the system continues to
                    >> grow and the refactoring efforts take longer. I have mentioned before
                    >> that there is a substantial technical debt in this system, and this
                    >> continues to be an issue. So, the end result is that our releases are
                    >> taking longer over time for an equivalent amount of functionality.

                    > I am wondering why it is that adding each new piece of functionality takes
                    > longer than the one before. I understand that the system as a whole is
                    > growing, but why are you having to refactor ever larger parts of the system
                    > with each addition? That implies to me that the new additions are not well
                    > designed, or perhaps the new applications are entirely incompatible with
                    > existing functionality.

                    If adding each new piece of functionality takes longer than the one
                    before, there is one and only one likely reason: we are making the
                    design worse with every feature.

                    Ron Jeffries
                    www.XProgramming.com
                    Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                    of his head. It is, as far as he knows, the only way of coming downstairs,
                    but sometimes he feels that there really is another way, if only he could
                    stop bumping for a moment and think of it. And then he feels that perhaps
                    there isn't. -- A. A. Milne
                  • news.gmane.org
                    Hi Dave Very good question! We ve recently moved from 3 code bases and fragmented teams to a single team and a gradually unifying single code base. We found
                    Message 9 of 9 , Nov 2, 2005
                    • 0 Attachment
                      Hi Dave

                      Very good question!

                      We've recently moved from 3 code bases and fragmented teams to a single
                      team and a gradually unifying single code base. We found that testing
                      how the different apps interfaced together was really a pain. We've
                      coined a term for our integrated architecture as well - a Monocoque
                      architecture.

                      The term Monocoque comes from car design You know how cars used to be
                      designed and made in separate chunks and then put together aftwards.
                      Well now they are designed and made as a unified structure. We use this
                      metaphore when explaining to management why we wanted to unify the teams.

                      There are many advantages for us in a Monocoque architecture, because
                      integration is more important than anything else. In our case, the whole
                      is definately greater than the sum of the parts. However if the separate
                      apps can stand by themselves and interfacing is only of limited benefit,
                      there may be less benefit from doing it the Monocoque way.

                      Daniel Poon


                      Dave Rooney wrote:

                      > I know that the conventional XP wisdom regarding frameworks is to build
                      > at least two applications and refactor the commonalities out into a
                      > framework to eliminate the duplication.
                      > However, what does the group think about dealing with the integration of
                      > multiple applications?
                      >
                      > At a current client, we have two different groups using two different
                      > approaches to delivering applications. The group with whom I currently
                      > work has a policy where any new functionality to support the business is
                      > integrated into the existing system. This results in a common code base
                      > for what can appear to the end user to be multiple applications. That's
                      > a "Good Thing". However, it does take a good chunk of work to properly
                      > integrate new functionality in terms of figuring out where to put
                      > things, and refactoring the code appropriately. Over the 2.5 year time
                      > period that this system has existed, it has taken gradually longer to
                      > integrate each new piece of functionality since the system continues to
                      > grow and the refactoring efforts take longer. I have mentioned before
                      > that there is a substantial technical debt in this system, and this
                      > continues to be an issue. So, the end result is that our releases are
                      > taking longer over time for an equivalent amount of functionality.
                      >
                      > Meanwhile, there is another team that uses a different approach. Their
                      > philosophy is to build small, simple applications to meet specific
                      > business criteria, and create interfaces between the applications as
                      > required. What I'm seeing from them is that they're able to react to
                      > business requests faster than we can, although they have a considerable
                      > amount of code duplicated between applications. While that team could be
                      > considered 'agile', they aren't specifically using an agile development
                      > method.
                      >
                      > So the question is, which approach is better? There are definitely
                      > merits to a single, integrated system, but in practice we've found that
                      > it certainly isn't easy to do and that our ability to deliver business
                      > value has diminished over time (based on our team's track record - we're
                      > still way ahead of most other teams). There are also merits to building
                      > small stovepipe applications with interfaces, but you then have to deal
                      > with duplicated code, possibly in multiple incompatible versions.
                      >
                      > I'm wondering if a hybrid approach would be better - build the stovepipe
                      > application to meet the immediate business need, build any required
                      > interfaces, but assume that over time that stovepipe will be integrated
                      > into the system to eliminate duplication.
                      >
                      > Thoughts?
                      >
                      > Dave Rooney
                      > Mayford Technologies
                      > http://www.mayford.ca
                      >
                      >
                      >
                      >
                      > To Post a message, send it to: extremeprogramming@...
                      >
                      > To Unsubscribe, send a blank message to:
                      extremeprogramming-unsubscribe@...
                      >
                      > ad-free courtesy of objectmentor.com
                    Your message has been successfully submitted and would be delivered to recipients shortly.