Loading ...
Sorry, an error occurred while loading the content.

Architecture for XP and scalable web sites

Expand Messages
  • Keith Richardson
    Hello, This forum has included several messages describing problems using XP with EJB, describing EJB as non-OO and other difficulties with the EJB
    Message 1 of 22 , Dec 31, 2000
    • 0 Attachment
      Hello,
      This forum has included several messages describing problems using XP
      with EJB, describing EJB as non-OO and other difficulties with the
      EJB architecture. A highly scalable web site must be able to automate
      persistance of session and environment data, allow deployment to be
      easily adjusted and many other features that EJBs (and more
      completely J2EE) provides. Are there other environments that have
      proven to be better for developing highly scalable solutions with XP
      or should I interpret these messages as saying that XP is not
      applicable for these needs? Any XP success stories in this area to be
      shared?
      Keith Richardson
    • Chad Fowler
      ... XP ... automate ... XP ... be ... I think the point of a lot of these messages is that EJB is not, at least at the beginning of a project, (in most cases)
      Message 2 of 22 , Jan 1, 2001
      • 0 Attachment
        --- In extremeprogramming@egroups.com, "Keith Richardson"
        <keith@s...> wrote:
        > Hello,
        > This forum has included several messages describing problems using
        XP
        > with EJB, describing EJB as non-OO and other difficulties with the
        > EJB architecture. A highly scalable web site must be able to
        automate
        > persistance of session and environment data, allow deployment to be
        > easily adjusted and many other features that EJBs (and more
        > completely J2EE) provides. Are there other environments that have
        > proven to be better for developing highly scalable solutions with
        XP
        > or should I interpret these messages as saying that XP is not
        > applicable for these needs? Any XP success stories in this area to
        be
        > shared?
        > Keith Richardson

        I think the point of a lot of these messages is that EJB is not, at
        least at the beginning of a project, (in most cases) the simplest
        thing that could possibly work. A well factored servlet application
        provides session persistence (not sure what you mean by "environment"
        in this context) and (being well factored) would also give
        flexibility for deployment options (things are logically decoupled,
        so they can be physically separated down the road).

        In the past, our developers were assuming that scalable-web-
        application == J2EE-all-the-way and just going with EJB from the
        start. This instroduced complexity in the code, the
        development/build process, and the application server deployment.
        This complexity invariably lead to various problems (more points of
        potential failure).

        The approach we've taken semi-recently with our developers is to
        say, "You are free to use EJB (and the rest of the J2EE baggage) if
        you can explain why you need it." What we've found so far is that no
        one has ever needed it, and we haven't had any related problems.
        (And, our applications are *so* much easier to deploy and maintain
        now!) In the mean time, we've been trying to keep things well
        factored, so they could be easily moved to the more complex solution
        should we ever hit the scalability barriers of this approach. My
        guess is that we won't ever get there.

        Of course, we're no amazon.com, but we get some pretty heavy traffic.

        Just as an addendum, here's my XP zealot answer:
        "must be able to automate persistance of session and environment
        data, allow deployment to be easily adjusted and many other features
        that EJBs (and more completely J2EE) provides" wasn't in my user
        stories. :)

        Chad
      • Paul Hodgetts
        ... Would you define what you mean by highly scalable? How often do you need to adjust the scale that the site must support? What are the increments of
        Message 3 of 22 , Jan 1, 2001
        • 0 Attachment
          Keith Richardson wrote:

          > A highly scalable web site

          Would you define what you mean by highly scalable? How often do
          you need to adjust the "scale" that the site must support? What
          are the increments of scale (how many scale targets)?

          My experience is that a site typically only goes through a few
          scale adjustments in its life cycle. It may start out with an
          initial deployment that supports a few thousand users, then as
          success warrants it gets bumped up to support some large target
          that marketing sets and probably never gets reached. If you're
          really successful, you may bump it once more to support a huge
          amount of users. The need to increase the scale is usually
          known before the system is maxed out, if we're paying attention
          to the usage stats and trends and doing some load testing.

          My point is that the ability to scale is important, but that
          the scale adjustments don't happen suddenly or very often. So
          if we're carrying along a lot of baggage to allow quick and
          frequent scale adjustments, it may not be worth the continuous
          cost of that baggage for the few times we bump the scale.

          > must be able to automate persistance of session and environment
          > data,

          How much automation do you want?

          There's that cost thing again. There's often more complexity
          with the "automated" persistent mechanisms, and I think many of
          them have additional build and deploy cycles that can limit our
          ability to rapidly and continuously integrate.

          > allow deployment to be easily adjusted and many other features
          > that EJBs (and more completely J2EE) provides.

          Similar to scaling, deployment adjustments usually won't happen
          all that often. In most sites, the deployment needs aren't all
          that complex and most everything gets deployed as just one or
          maybe a few sets of components. It's only when we get to much
          larger sites with a wider range of deployed components that the
          deployments get complicated. When the site's in development,
          there's probably more thrashing with moving things around. This
          stabilizes quite a bit as time goes on. Again, is it worth a lot
          of day-to-day baggage for the few times that components get moved
          around in a deployment?

          > Are there other environments that have proven to be better for
          > developing highly scalable solutions with XP or should I
          > interpret these messages as saying that XP is not applicable for
          > these needs? Any XP success stories in this area to be shared?

          I don't think any of the distributed environments (J2EE, MTS,
          CORBA) are incompatible with XP per se. XP would ask us to
          implement the simplest solution to meet the current requirements,
          and to evolve that solution as the requirements change. I'm not
          sure incrementally working from the initial requirements to an
          eventual solution would necessarily lead us to choose these
          technologies, or if we'd end up with something else entirely.

          So far our team is learning that it's best to use good design
          principles that reduce the coupling and allow the specific
          implementations of the various technologies to be changed without
          thrashing the whole system. Then you've got flexibility and you
          don't need to commit the project to a specific technology early.

          Most of these technologies have a lot of overhead in terms of
          building, deploying, and starting/stopping the environments. This
          cuts into the speed at which we can integrate new code. IMHO,
          it's extremely important to be able to test the core business
          classes outside the environment, and only pull the environment in
          when needed for more comprehensive testing. Good decoupled
          designs that are kept that way through continuous refactoring help.

          We made the mistake of coupling our design too closely to the
          technologies and now we have to fix that. We also relied a lot
          on EJB entity beans, and they have not lived up to our needs
          (performance problems, added complexity, inflexibility of the
          persistent object model). Our primary symptoms are that we
          can't implement adequate unit testing, and our code/build/test
          cycles are too long for continuous integration.

          I'm starting to be convinced that using XP principles from the
          start on a web site development project would probably lead to
          a surprisingly simple solution. Maybe just some front end
          servlets or JSPs, perhaps using XML and XSLT, accessing the
          persistent data via a simple mapping layer. If I had a more
          open mind, maybe I'd end up with a non-Java based solution. ;-)

          I see no reason a simple solution like this can't be scalable,
          deployable, and all the other -ilities you want from a site.
          It doesn't sound as cool as a grand J2EE implementation, but it
          might get a whole lot of business value implemented a whole lot
          faster.

          -Paul Hodgetts
        • John Brewer
          ... Don t believe all of Sun s hype. EJBs aren t done yet. In particular, the current state of entity beans leaves lots to be desired. I ve heard of at least
          Message 4 of 22 , Jan 1, 2001
          • 0 Attachment
            --- In extremeprogramming@egroups.com, "Keith Richardson" <keith@s...>
            wrote:
            > A highly scalable web site must be able to automate
            > persistance of session and environment data, allow deployment to be
            > easily adjusted and many other features that EJBs (and more
            > completely J2EE) provides.

            Don't believe all of Sun's hype. EJBs aren't done yet. In
            particular, the current state of entity beans leaves lots to be
            desired.

            I've heard of at least one XP project that ended up removing the
            entity beans from their system because of performance problems.
            Fortunately, they had the well-factored code and corresponding tests
            to make that possible. I believe they eventually eliminated all the
            fancy app-server stuff from their code, allowing them to deploy using
            an inexpensive servlet container.

            John Brewer
            Jera Design
          • Keith Richardson
            ... ... be ... tests ... the ... using ... I have also heard of some projects having these problems. I would love to hear about architectures that
            Message 5 of 22 , Jan 1, 2001
            • 0 Attachment
              --- In extremeprogramming@egroups.com, "John Brewer" <jbrewer@j...>
              wrote:
              > --- In extremeprogramming@egroups.com, "Keith Richardson"
              <keith@s...>
              > wrote:
              > > A highly scalable web site must be able to automate
              > > persistance of session and environment data, allow deployment to
              be
              > > easily adjusted and many other features that EJBs (and more
              > > completely J2EE) provides.
              >
              > Don't believe all of Sun's hype. EJBs aren't done yet. In
              > particular, the current state of entity beans leaves lots to be
              > desired.
              >
              > I've heard of at least one XP project that ended up removing the
              > entity beans from their system because of performance problems.
              > Fortunately, they had the well-factored code and corresponding
              tests
              > to make that possible. I believe they eventually eliminated all
              the
              > fancy app-server stuff from their code, allowing them to deploy
              using
              > an inexpensive servlet container.
              >

              I have also heard of some projects having these problems. I would
              love to hear about architectures that have worked. I will be
              developing Web site frameworks that will be deployed by our clients
              so the solution must scale to fit a wide range of clients. I have
              heard many success stories of J2EE/WebLogic scaling exceptionally
              well and any alternative architecture must be just as good.
              Suggestions?
              Keith Richardson
            • Keith Richardson
              We are developing a Web application framework for a specialized vertical market that will be deployed in our client s environments. The solution must scale to
              Message 6 of 22 , Jan 1, 2001
              • 0 Attachment
                We are developing a Web application framework for a specialized
                vertical market that will be deployed in our client's environments.
                The solution must scale to fit a wide range of clients. We can not
                accurately predict the way future clients will use the solution
                (think "shrink wrapped software"). The most important customers are
                the biggest ones and we can not afford to find that the solution does
                not scale to meet the requirements when we get interest from a new
                major client. i.e. We will get a sudden major jump in performance
                requirements is a new customer comes along who pushes the limits of
                what we are doing.

                For these reasons I consider scalability to be a fundamental
                requirement. We are not in a position to develop to meet just the
                performance needs in front of us right now so I beleive we must
                select technologies to give us very flexible deployment. Before
                considering XP I had selected J2EE/Weblogic as a good solution. Other
                related complanies in our field are using the same environment which
                is a double plus - the technology is proven and we have the
                opportunity to share code in the future. (i.e. they could become
                customers by deploying components of our software.)

                Automated persistence is a plus because we have no way of predicting
                what state the users will be in when the system gets an unexpected
                burst of heavy use. Without automated persistence, we would have to
                do all the coding to make decisions on which session components get
                swapped out when things get busy. It is easy building a Web site but
                not easy building one to handle bursts of use that are orders of
                magnitude higher than average. Everything else being equal, I would
                prefer the deployment architecture to take care of this. This again
                leads to the J2EE/Weblogic option.

                Now I am trying to decide how XP fits in this environment of whether
                an alternative should be considered to better fit XP. I also came to
                your conclusion that XP should lead to simple solutions. There seems
                no reason why J2EE can't be kept simple. Or is there?

                Keith Richardson

                --- In extremeprogramming@egroups.com, Paul Hodgetts <prh@z...> wrote:
                > Keith Richardson wrote:
                >
                > > A highly scalable web site
                >
                > Would you define what you mean by highly scalable? How often do
                > you need to adjust the "scale" that the site must support? What
                > are the increments of scale (how many scale targets)?
                >
                > My experience is that a site typically only goes through a few
                > scale adjustments in its life cycle. It may start out with an
                > initial deployment that supports a few thousand users, then as
                > success warrants it gets bumped up to support some large target
                > that marketing sets and probably never gets reached. If you're
                > really successful, you may bump it once more to support a huge
                > amount of users. The need to increase the scale is usually
                > known before the system is maxed out, if we're paying attention
                > to the usage stats and trends and doing some load testing.
                >
                > My point is that the ability to scale is important, but that
                > the scale adjustments don't happen suddenly or very often. So
                > if we're carrying along a lot of baggage to allow quick and
                > frequent scale adjustments, it may not be worth the continuous
                > cost of that baggage for the few times we bump the scale.
                >
                > > must be able to automate persistance of session and environment
                > > data,
                >
                > How much automation do you want?
                >
                > There's that cost thing again. There's often more complexity
                > with the "automated" persistent mechanisms, and I think many of
                > them have additional build and deploy cycles that can limit our
                > ability to rapidly and continuously integrate.
                >
                > > allow deployment to be easily adjusted and many other features
                > > that EJBs (and more completely J2EE) provides.
                >
                > Similar to scaling, deployment adjustments usually won't happen
                > all that often. In most sites, the deployment needs aren't all
                > that complex and most everything gets deployed as just one or
                > maybe a few sets of components. It's only when we get to much
                > larger sites with a wider range of deployed components that the
                > deployments get complicated. When the site's in development,
                > there's probably more thrashing with moving things around. This
                > stabilizes quite a bit as time goes on. Again, is it worth a lot
                > of day-to-day baggage for the few times that components get moved
                > around in a deployment?
                >
                > > Are there other environments that have proven to be better for
                > > developing highly scalable solutions with XP or should I
                > > interpret these messages as saying that XP is not applicable for
                > > these needs? Any XP success stories in this area to be shared?
                >
                > I don't think any of the distributed environments (J2EE, MTS,
                > CORBA) are incompatible with XP per se. XP would ask us to
                > implement the simplest solution to meet the current requirements,
                > and to evolve that solution as the requirements change. I'm not
                > sure incrementally working from the initial requirements to an
                > eventual solution would necessarily lead us to choose these
                > technologies, or if we'd end up with something else entirely.
                >
                > So far our team is learning that it's best to use good design
                > principles that reduce the coupling and allow the specific
                > implementations of the various technologies to be changed without
                > thrashing the whole system. Then you've got flexibility and you
                > don't need to commit the project to a specific technology early.
                >
                > Most of these technologies have a lot of overhead in terms of
                > building, deploying, and starting/stopping the environments. This
                > cuts into the speed at which we can integrate new code. IMHO,
                > it's extremely important to be able to test the core business
                > classes outside the environment, and only pull the environment in
                > when needed for more comprehensive testing. Good decoupled
                > designs that are kept that way through continuous refactoring help.
                >
                > We made the mistake of coupling our design too closely to the
                > technologies and now we have to fix that. We also relied a lot
                > on EJB entity beans, and they have not lived up to our needs
                > (performance problems, added complexity, inflexibility of the
                > persistent object model). Our primary symptoms are that we
                > can't implement adequate unit testing, and our code/build/test
                > cycles are too long for continuous integration.
                >
                > I'm starting to be convinced that using XP principles from the
                > start on a web site development project would probably lead to
                > a surprisingly simple solution. Maybe just some front end
                > servlets or JSPs, perhaps using XML and XSLT, accessing the
                > persistent data via a simple mapping layer. If I had a more
                > open mind, maybe I'd end up with a non-Java based solution. ;-)
                >
                > I see no reason a simple solution like this can't be scalable,
                > deployable, and all the other -ilities you want from a site.
                > It doesn't sound as cool as a grand J2EE implementation, but it
                > might get a whole lot of business value implemented a whole lot
                > faster.
                >
                > -Paul Hodgetts
              • Keith Richardson
                ... application ... by environment ... no ... solution ... traffic. ... features ... How would you have handled the user story: Must be able to scale
                Message 7 of 22 , Jan 1, 2001
                • 0 Attachment
                  --- In extremeprogramming@egroups.com, "Chad Fowler"
                  <chadfowler@y...> wrote:
                  > I think the point of a lot of these messages is that EJB is not, at
                  > least at the beginning of a project, (in most cases) the simplest
                  > thing that could possibly work. A well factored servlet
                  application
                  > provides session persistence (not sure what you mean
                  by "environment"
                  > in this context) and (being well factored) would also give
                  > flexibility for deployment options (things are logically decoupled,
                  > so they can be physically separated down the road).
                  >
                  > In the past, our developers were assuming that scalable-web-
                  > application == J2EE-all-the-way and just going with EJB from the
                  > start. This instroduced complexity in the code, the
                  > development/build process, and the application server deployment.
                  > This complexity invariably lead to various problems (more points of
                  > potential failure).
                  >
                  > The approach we've taken semi-recently with our developers is to
                  > say, "You are free to use EJB (and the rest of the J2EE baggage) if
                  > you can explain why you need it." What we've found so far is that
                  no
                  > one has ever needed it, and we haven't had any related problems.
                  > (And, our applications are *so* much easier to deploy and maintain
                  > now!) In the mean time, we've been trying to keep things well
                  > factored, so they could be easily moved to the more complex
                  solution
                  > should we ever hit the scalability barriers of this approach. My
                  > guess is that we won't ever get there.
                  >
                  > Of course, we're no amazon.com, but we get some pretty heavy
                  traffic.
                  >
                  > Just as an addendum, here's my XP zealot answer:
                  > "must be able to automate persistance of session and environment
                  > data, allow deployment to be easily adjusted and many other
                  features
                  > that EJBs (and more completely J2EE) provides" wasn't in my user
                  > stories. :)
                  >
                  > Chad

                  How would you have handled the user story: "Must be able to scale
                  quickly to unpredictable levels when a new client wants to deploy in
                  a high load environment"? Maybe there is a good reason by this is a
                  bad user story but I am having a hard time not having this need lead
                  to a technology selection.
                  Keith Richardson
                • Ron Jeffries
                  ... Well, this is a copout, but that story can t be estimated (at least not by me). I d have to break it down into stories like must be able to run 100 hits
                  Message 8 of 22 , Jan 1, 2001
                  • 0 Attachment
                    At 11:27 PM 1/1/2001 +0000, it seemed like Keith Richardson wrote:
                    >How would you have handled the user story: "Must be able to scale
                    >quickly to unpredictable levels when a new client wants to deploy in
                    >a high load environment"? Maybe there is a good reason by this is a
                    >bad user story but I am having a hard time not having this need lead
                    >to a technology selection.

                    Well, this is a copout, but that story can't be estimated (at least not by
                    me). I'd have to break it down into stories like "must be able to run 100
                    hits per second on a Pentium IV" or something.

                    Then I'd have to do some experiments. This process might look almost
                    exactly like technology selection.

                    Ronald E Jeffries
                    http://www.XProgramming.com
                    http://www.objectmentor.com
                  • Chad Fowler
                    ... at ... in ... lead ... Perhaps, your project doesn t fall into the in most cases category. If it were me, I would choose the simplest thing that I
                    Message 9 of 22 , Jan 1, 2001
                    • 0 Attachment
                      --- In extremeprogramming@egroups.com, "Keith Richardson"
                      <keith@s...> wrote:
                      > --- In extremeprogramming@egroups.com, "Chad Fowler"
                      > <chadfowler@y...> wrote:
                      > > I think the point of a lot of these messages is that EJB is not,
                      at
                      > > least at the beginning of a project, (in most cases) the simplest
                      > > thing that could possibly work.

                      > How would you have handled the user story: "Must be able to scale
                      > quickly to unpredictable levels when a new client wants to deploy
                      in
                      > a high load environment"? Maybe there is a good reason by this is a
                      > bad user story but I am having a hard time not having this need
                      lead
                      > to a technology selection.

                      Perhaps, your project doesn't fall into the "in most cases"
                      category. If it were me, I would choose the simplest thing that I
                      *believe* could possibly work and follow up with a performance spike
                      using load testing software. Of course, as Ron mentioned in a
                      previous response, words like "unpredictable" don't lend themselves
                      to estimation. I would first try to get a solid idea of
                      what "unpredictable" might mean.
                    • Keith Richardson
                      ... in ... lead ... not by ... run 100 ... almost ... I think I am faced with having to take a copout of some form. We will be creating solutions that are
                      Message 10 of 22 , Jan 1, 2001
                      • 0 Attachment
                        --- In extremeprogramming@egroups.com, Ron Jeffries
                        <ronjeffries@a...> wrote:
                        > At 11:27 PM 1/1/2001 +0000, it seemed like Keith Richardson wrote:
                        > >How would you have handled the user story: "Must be able to scale
                        > >quickly to unpredictable levels when a new client wants to deploy
                        in
                        > >a high load environment"? Maybe there is a good reason by this is a
                        > >bad user story but I am having a hard time not having this need
                        lead
                        > >to a technology selection.
                        >
                        > Well, this is a copout, but that story can't be estimated (at least
                        not by
                        > me). I'd have to break it down into stories like "must be able to
                        run 100
                        > hits per second on a Pentium IV" or something.
                        >
                        > Then I'd have to do some experiments. This process might look
                        almost
                        > exactly like technology selection.
                        >
                        > Ronald E Jeffries
                        > http://www.XProgramming.com
                        > http://www.objectmentor.com

                        I think I am faced with having to take a copout of some form. We will
                        be creating solutions that are installed by our clients and used by
                        their customers. Some clients are large with jobs running through
                        hundreds of plants that are accessed though a single Web site. I have
                        absolutely no idea how often their customers will be accessing the
                        site - the clients don't know either. Trying to put a number of this
                        would be a wild guess at best. The worst that can happen if the most
                        scalable technology is selected is the project takes longer to
                        implement and the deployment costs are slightly higher. The worst
                        that can happen if we go for lightweight solutions is that we find
                        the system is unusable when it eventually gets deployed. That would
                        be bad!

                        There have been several messages in this forum describing situations
                        where EJB was found to be overkill and unnecessary. I am worried that
                        selecting another technology could leave me doing major rework
                        against a deployment deadline when I run in to the eventual
                        performance bottleneck. What success have other Web developers had in
                        passing performance limits?
                        Keith Richardson
                      • Dossy
                        ... This does sound like a big cop-out. Can we not apply Yesterday s Weather to this problem? Do you have ANY customers? Or are you still in the haven t
                        Message 11 of 22 , Jan 1, 2001
                        • 0 Attachment
                          On 2001.01.02, Keith Richardson <keith@...> wrote:
                          > I think I am faced with having to take a copout of some form. We will
                          > be creating solutions that are installed by our clients and used by
                          > their customers. Some clients are large with jobs running through
                          > hundreds of plants that are accessed though a single Web site. I have
                          > absolutely no idea how often their customers will be accessing the
                          > site - the clients don't know either. Trying to put a number of this
                          > would be a wild guess at best.

                          This does sound like a big cop-out. Can we not apply Yesterday's
                          Weather to this problem?

                          Do you have ANY customers? Or are you still in the "haven't sold
                          product" stage and we're just doing a lot of what-if? (If your
                          answers are "no" and "yes", then I'm tempted to say YAGNI. ;-) )

                          If you already have a customer, can you not use them as a "model"
                          for other customers in their size/scale/class, at least to create
                          estimates that are even slightly better than WAGs?

                          Can you not use measurements from smaller customers to approximate
                          what measurements might look like for larger customers? Can you
                          not create a spike to at least verify that these approximations
                          are "close enough"? (Complexologists would suggest building
                          simulations - I tend to agree.)

                          It might be very costly to build a worthwhile simulation of a
                          theoretical "huge" customer, but with the kind of dollars on
                          the line for that kind of customer, wouldn't it be worth it?
                          What would be more costly, the simulation or the loss of the
                          customer because you didn't fully understand your own product
                          that you're selling?

                          > The worst that can happen if the most
                          > scalable technology is selected is the project takes longer to
                          > implement and the deployment costs are slightly higher. The worst
                          > that can happen if we go for lightweight solutions is that we find
                          > the system is unusable when it eventually gets deployed. That would
                          > be bad!

                          If you're really doing XP, the cost of change curve should be
                          "flat enough" such that moving from a lightweight solution to a
                          "scalable technology" shouldn't have an oppressive cost.

                          If you implement the lightweight solution poorly, sure, it'll
                          be that much tougher to enhance the system to meet scalability
                          needs later on. But that's not a technology selection problem...

                          Make the system usable when it's deployed, and make it deployable
                          ASAP for the customers you DO know about. When a big fat customer
                          comes along, the money they should pay should justify any serious
                          enhancements to the system, and as long as you've kept the system
                          as simple as possible, enhancing where it's necessary (and only
                          where it's necessary) shouldn't be too hard.


                          - Dossy

                          --
                          Dossy Shiobara mail: dossy@...
                          Panoptic Computer Network web: http://www.panoptic.com/
                        • Ron Jeffries
                          ... With all due respect, I believe the above to be almost exactly incorrect. The worst that can happen if a big technology is chosen is that you get to market
                          Message 12 of 22 , Jan 1, 2001
                          • 0 Attachment
                            At 02:55 AM 1/2/2001 +0000, it seemed like Keith Richardson wrote:
                            >The worst that can happen if the most
                            >scalable technology is selected is the project takes longer to
                            >implement and the deployment costs are slightly higher. The worst
                            >that can happen if we go for lightweight solutions is that we find
                            >the system is unusable when it eventually gets deployed. That would
                            >be bad!

                            With all due respect, I believe the above to be almost exactly incorrect.

                            The worst that can happen if a big technology is chosen is that you get to
                            market late, lose market share, and don't learn what people really want
                            until it is too late. The worst that can happen if you go for lightweight
                            solutions is that you have to beef it up upon deployment. But you find that
                            out about a week after you start.

                            Flexibility is better than planning, every time.

                            Ronald E Jeffries
                            http://www.XProgramming.com
                            http://www.objectmentor.com
                          • Glen B. Alleman
                            Flexibility versus planning? ... critical (realtime shop floor control) or is it informational to a non-critical set of users? That is a big range of
                            Message 13 of 22 , Jan 1, 2001
                            • 0 Attachment
                              Flexibility versus planning?

                              Here's some questions that may help clarify the situation:

                              >>What is the context of the problem here?
                              >>Where are the boundaries of your recommendation here?
                              >>Where in the overall scheme of things does this system fit? Is it mission
                              critical (realtime shop floor control) or is it informational to a
                              non-critical set of users? That is a big range of "domain."

                              The problem described by the original poster was a bit vague I agree. The
                              manufacturing domain problems of "scalability estimates" are common,
                              serious, and at the same time unknown without a lot of leg work. In a large
                              manufacturing systems, "massive customization" is supported by a
                              "configurator" based BOM system. See http://www.configsc.com/logia2.htm as
                              an example of such a system. This is not the best, there are others (I2
                              being a better one). This process can whipsaw the transaction rates by 2 to
                              3 orders of magnitude when a simple "build to order" configuration is
                              changed by marketing in an attempt to reposition a product line. The sizing
                              impacts on this type of plant or shop floor system are a challenge for the
                              experts, let along someone just getting started in the development of a
                              plant data management system.

                              This fellow needs to perform some analysis to determine the boundaries of
                              the problem before embarking on ANY development method. He needs to get a
                              strategy of how to scale the system "if and when" scaling is needed. If it
                              is not needed, fine, but if it is then the system must be capable of
                              performing this scaling without disruption to the ongoing business.

                              If it is an on going business, then there are some CRUD stats somewhere,
                              even if they have to be gathered by hand. Those then are used to "size" the
                              solutions and determine the underlying technology needs. This is really
                              simple IT Strategy stuff. There is definitely missing information in this
                              case, which will most likely cause problems down stream. He's in no position
                              to pick any technology without first understanding the processes and the
                              data that they touch.

                              Glen B Alleman
                              Niwot Ridge Consulting

                              "Every even number greater than 2 is the sum of two primes"


                              >-----Original Message-----
                              >From: Ron Jeffries [mailto:ronjeffries@...]
                              >Sent: Monday, January 01, 2001 8:28 PM
                              >To: extremeprogramming@egroups.com
                              >Subject: Re: [XP] Re: Architecture for XP and scalable web sites
                              >
                              >
                              >At 02:55 AM 1/2/2001 +0000, it seemed like Keith Richardson wrote:
                              >>The worst that can happen if the most
                              >>scalable technology is selected is the project takes longer to
                              >>implement and the deployment costs are slightly higher. The worst
                              >>that can happen if we go for lightweight solutions is that we find
                              >>the system is unusable when it eventually gets deployed. That would
                              >>be bad!
                              >
                              >With all due respect, I believe the above to be almost exactly incorrect.
                              >
                              >The worst that can happen if a big technology is chosen is that you get to
                              >market late, lose market share, and don't learn what people really want
                              >until it is too late. The worst that can happen if you go for lightweight
                              >solutions is that you have to beef it up upon deployment. But you
                              >find that
                              >out about a week after you start.
                              >
                              >Flexibility is better than planning, every time.
                              >
                              >Ronald E Jeffries
                              >
                            • Patrick Logan
                              ... I would notice that this story is ambiguous.
                              Message 14 of 22 , Jan 1, 2001
                              • 0 Attachment
                                --- "Keith Richardson" <keith@s...> wrote:

                                > How would you have handled the user story: "Must be able to
                                > scale quickly to unpredictable levels when a new client wants
                                > to deploy in a high load environment"?

                                I would notice that this story is ambiguous.
                              • Laurent Bossavit
                                ... Wouldn t it be worse if you selected the most scalable technology, took longer to implement the project, *and* found the system didn t respond well to high
                                Message 15 of 22 , Jan 2, 2001
                                • 0 Attachment
                                  > The worst that can happen if the most scalable technology is
                                  > selected is the project takes longer to implement and the
                                  > deployment costs are slightly higher. The worst that can happen if
                                  > we go for lightweight solutions is that we find the system is
                                  > unusable when it eventually gets deployed. That would be bad!

                                  Wouldn't it be worse if you selected the most scalable technology,
                                  took longer to implement the project, *and* found the system didn't
                                  respond well to high loads when deployed ?

                                  Do you have any guarantee that won't happen ? What steps can be
                                  taken to prevent this happening ?


                                  ========================================
                                  We aim to make simple things simple and
                                  complex things possible.
                                  ========================================
                                  Laurent Bossavit - Software Architect
                                  >>> laurent.bossavit@... <<<
                                  >>> 06 68 15 11 44 <<<
                                  >> ICQ#39281367 <<
                                  Agence Bless http://www.agencebless.com/
                                  ========================================
                                • Keith Richardson
                                  ... What a great forum - I posted a message at 10pm and got many great responses by 7am!! Either you guys never sleep or most of you are not is the EST time
                                  Message 16 of 22 , Jan 2, 2001
                                  • 0 Attachment
                                    --- In extremeprogramming@egroups.com, "Laurent Bossavit"
                                    <laurent.bossavit@a...> wrote:
                                    > > The worst that can happen if the most scalable technology is
                                    > > selected is the project takes longer to implement and the
                                    > > deployment costs are slightly higher. The worst that can happen if
                                    > > we go for lightweight solutions is that we find the system is
                                    > > unusable when it eventually gets deployed. That would be bad!
                                    >
                                    > Wouldn't it be worse if you selected the most scalable technology,
                                    > took longer to implement the project, *and* found the system didn't
                                    > respond well to high loads when deployed ?
                                    >
                                    > Do you have any guarantee that won't happen ? What steps can be
                                    > taken to prevent this happening ?

                                    What a great forum - I posted a message at 10pm and got many great
                                    responses by 7am!! Either you guys never sleep or most of you are not
                                    is the EST time zone!

                                    The clear consensus is to keep the technology simple. Now to pick a
                                    robust and simple technology that works well with XP. Java is a much
                                    cleaner environment than ActiveX which would suggest not using MS-
                                    based development environment for IIS. A new question: what
                                    combination of development tools and deployment environment is best
                                    for a Web application built using XP? It should have a clean object
                                    model and a build environment that allows for rapid development cycle
                                    times. It must also be robust and a good performer because I don't
                                    want to be kludging around limitations or speed bottlenecks. What is
                                    being used for XP and how is it working?
                                    Keith Richardson
                                  • Chad Fowler
                                    ... cycle ... I would suggest that keeping your technology choice as open as possible is a good plan. This enables you to start with simplicity and grow when
                                    Message 17 of 22 , Jan 2, 2001
                                    • 0 Attachment
                                      >
                                      > The clear consensus is to keep the technology simple. Now to pick a
                                      > robust and simple technology that works well with XP. Java is a much
                                      > cleaner environment than ActiveX which would suggest not using MS-
                                      > based development environment for IIS. A new question: what
                                      > combination of development tools and deployment environment is best
                                      > for a Web application built using XP? It should have a clean object
                                      > model and a build environment that allows for rapid development
                                      cycle
                                      > times. It must also be robust and a good performer because I don't
                                      > want to be kludging around limitations or speed bottlenecks. What is
                                      > being used for XP and how is it working?
                                      > Keith Richardson

                                      I would suggest that keeping your technology choice as open as
                                      possible is a good plan. This enables you to start with simplicity
                                      and grow when the need arises with less investment than a proprietary
                                      alternative would afford. We are successfully using and deploying
                                      Enhydra (http://www.enhydra.org) in a change-heavy iterative
                                      development environment. We have been very happy with its performance
                                      and flexibility (as well as its price--Open Source).
                                    • Stefan Schmiedl
                                      They work 40 hour weeks and hence have lots of free time on their hands.... :-) Stefan +---------+------------------------- ...
                                      Message 18 of 22 , Jan 2, 2001
                                      • 0 Attachment
                                        They work 40 hour weeks and hence have lots of free time on their
                                        hands.... :-)

                                        Stefan

                                        +---------+-------------------------
                                        | from | Keith Richardson <keith@...>
                                        | to | extremeprogramming@egroups.com <extremeprogramming@egroups.com>
                                        | date | 02.01.2001 16:13
                                        | subject | [XP] Re: Architecture for XP and scalable web sites
                                        +---------+-------------------------

                                        K> What a great forum - I posted a message at 10pm and got many great
                                        K> responses by 7am!! Either you guys never sleep or most of you are not
                                        K> is the EST time zone!
                                      • astl@home.com
                                        ... using ... the ... be ... to ... application ... by environment ... no ... solution ... traffic. ... features ... Do the user stories say don t lose my
                                        Message 19 of 22 , Jan 12, 2001
                                        • 0 Attachment
                                          --- In extremeprogramming@egroups.com, "Chad Fowler"
                                          <chadfowler@y...> wrote:
                                          > --- In extremeprogramming@egroups.com, "Keith Richardson"
                                          > <keith@s...> wrote:
                                          > > Hello,
                                          > > This forum has included several messages describing problems
                                          using
                                          > XP
                                          > > with EJB, describing EJB as non-OO and other difficulties with
                                          the
                                          > > EJB architecture. A highly scalable web site must be able to
                                          > automate
                                          > > persistance of session and environment data, allow deployment to
                                          be
                                          > > easily adjusted and many other features that EJBs (and more
                                          > > completely J2EE) provides. Are there other environments that have
                                          > > proven to be better for developing highly scalable solutions with
                                          > XP
                                          > > or should I interpret these messages as saying that XP is not
                                          > > applicable for these needs? Any XP success stories in this area
                                          to
                                          > be
                                          > > shared?
                                          > > Keith Richardson
                                          >
                                          > I think the point of a lot of these messages is that EJB is not, at
                                          > least at the beginning of a project, (in most cases) the simplest
                                          > thing that could possibly work. A well factored servlet
                                          application
                                          > provides session persistence (not sure what you mean
                                          by "environment"
                                          > in this context) and (being well factored) would also give
                                          > flexibility for deployment options (things are logically decoupled,
                                          > so they can be physically separated down the road).
                                          >
                                          > In the past, our developers were assuming that scalable-web-
                                          > application == J2EE-all-the-way and just going with EJB from the
                                          > start. This instroduced complexity in the code, the
                                          > development/build process, and the application server deployment.
                                          > This complexity invariably lead to various problems (more points of
                                          > potential failure).
                                          >
                                          > The approach we've taken semi-recently with our developers is to
                                          > say, "You are free to use EJB (and the rest of the J2EE baggage) if
                                          > you can explain why you need it." What we've found so far is that
                                          no
                                          > one has ever needed it, and we haven't had any related problems.
                                          > (And, our applications are *so* much easier to deploy and maintain
                                          > now!) In the mean time, we've been trying to keep things well
                                          > factored, so they could be easily moved to the more complex
                                          solution
                                          > should we ever hit the scalability barriers of this approach. My
                                          > guess is that we won't ever get there.
                                          >
                                          > Of course, we're no amazon.com, but we get some pretty heavy
                                          traffic.
                                          >
                                          > Just as an addendum, here's my XP zealot answer:
                                          > "must be able to automate persistance of session and environment
                                          > data, allow deployment to be easily adjusted and many other
                                          features
                                          > that EJBs (and more completely J2EE) provides" wasn't in my user
                                          > stories. :)
                                          >
                                          > Chad

                                          Do the user stories say "don't lose my data that I am entering just
                                          because you want to update the servers?"

                                          Ken.
                                        • Alan Francis
                                          ... ... Can we PLEASE try to keep the copied text to a minimum ? Even if the two new lines are the most
                                          Message 20 of 22 , Jan 12, 2001
                                          • 0 Attachment
                                            --- In extremeprogramming@egroups.com, astl@h... wrote:
                                            <a zillion lines of copied message>
                                            <followed by....>
                                            > Do the user stories say "don't lose my data that I am entering just
                                            > because you want to update the servers?"

                                            Can we PLEASE try to keep the copied text to a minimum ? Even if the
                                            two new lines are the most insightful piece of observation yet, it's
                                            still anooying to get massive emails full of copied stuff.

                                            Maybe OMI could host an extremeprogramming wiki so conversations
                                            could be conversations ? :-)

                                            Alan
                                          • Chad Fowler
                                            ... If they do, then it s time to find the simplest solution that can satisfy the story--again, not limited to EJB (nor very likely to be).
                                            Message 21 of 22 , Jan 13, 2001
                                            • 0 Attachment
                                              > > Just as an addendum, here's my XP zealot answer:
                                              > > "must be able to automate persistance of session and environment
                                              > > data, allow deployment to be easily adjusted and many other
                                              > features
                                              > > that EJBs (and more completely J2EE) provides" wasn't in my user
                                              > > stories. :)
                                              > >
                                              > Do the user stories say "don't lose my data that I am entering just
                                              > because you want to update the servers?"
                                              >
                                              > Ken.

                                              If they do, then it's time to find the simplest solution that can
                                              satisfy the story--again, not limited to EJB (nor very likely to be).
                                            • Erik Meade
                                              I mentioned this to Ward a few weeks ago, he said that Extreme Programming is on topic for his (The) Wiki http://www.c2.com/cgi/wiki?FrontPage -- Erik Meade
                                              Message 22 of 22 , Jan 17, 2001
                                              • 0 Attachment
                                                I mentioned this to Ward a few weeks ago, he said that Extreme
                                                Programming is on topic for his (The) Wiki
                                                http://www.c2.com/cgi/wiki?FrontPage

                                                --
                                                Erik Meade emeade@...
                                                Senior Consultant Object Mentor, Inc.
                                                http://www.junit.org


                                                -----Original Message-----
                                                From: "Alan Francis" <alan@...>
                                                Subject: Re: Architecture for XP and scalable web sites

                                                Maybe OMI could host an extremeprogramming wiki so conversations
                                                could be conversations ? :-)

                                                Alan
                                              Your message has been successfully submitted and would be delivered to recipients shortly.