Loading ...
Sorry, an error occurred while loading the content.

Re: Architecture for XP and scalable web sites

Expand Messages
  • Chad Fowler
    ... at ... in ... lead ... Perhaps, your project doesn t fall into the in most cases category. If it were me, I would choose the simplest thing that I
    Message 1 of 22 , Jan 1, 2001
    • 0 Attachment
      --- In extremeprogramming@egroups.com, "Keith Richardson"
      <keith@s...> wrote:
      > --- In extremeprogramming@egroups.com, "Chad Fowler"
      > <chadfowler@y...> wrote:
      > > I think the point of a lot of these messages is that EJB is not,
      at
      > > least at the beginning of a project, (in most cases) the simplest
      > > thing that could possibly work.

      > How would you have handled the user story: "Must be able to scale
      > quickly to unpredictable levels when a new client wants to deploy
      in
      > a high load environment"? Maybe there is a good reason by this is a
      > bad user story but I am having a hard time not having this need
      lead
      > to a technology selection.

      Perhaps, your project doesn't fall into the "in most cases"
      category. If it were me, I would choose the simplest thing that I
      *believe* could possibly work and follow up with a performance spike
      using load testing software. Of course, as Ron mentioned in a
      previous response, words like "unpredictable" don't lend themselves
      to estimation. I would first try to get a solid idea of
      what "unpredictable" might mean.
    • Keith Richardson
      ... in ... lead ... not by ... run 100 ... almost ... I think I am faced with having to take a copout of some form. We will be creating solutions that are
      Message 2 of 22 , Jan 1, 2001
      • 0 Attachment
        --- In extremeprogramming@egroups.com, Ron Jeffries
        <ronjeffries@a...> wrote:
        > At 11:27 PM 1/1/2001 +0000, it seemed like Keith Richardson wrote:
        > >How would you have handled the user story: "Must be able to scale
        > >quickly to unpredictable levels when a new client wants to deploy
        in
        > >a high load environment"? Maybe there is a good reason by this is a
        > >bad user story but I am having a hard time not having this need
        lead
        > >to a technology selection.
        >
        > Well, this is a copout, but that story can't be estimated (at least
        not by
        > me). I'd have to break it down into stories like "must be able to
        run 100
        > hits per second on a Pentium IV" or something.
        >
        > Then I'd have to do some experiments. This process might look
        almost
        > exactly like technology selection.
        >
        > Ronald E Jeffries
        > http://www.XProgramming.com
        > http://www.objectmentor.com

        I think I am faced with having to take a copout of some form. We will
        be creating solutions that are installed by our clients and used by
        their customers. Some clients are large with jobs running through
        hundreds of plants that are accessed though a single Web site. I have
        absolutely no idea how often their customers will be accessing the
        site - the clients don't know either. Trying to put a number of this
        would be a wild guess at best. The worst that can happen if the most
        scalable technology is selected is the project takes longer to
        implement and the deployment costs are slightly higher. The worst
        that can happen if we go for lightweight solutions is that we find
        the system is unusable when it eventually gets deployed. That would
        be bad!

        There have been several messages in this forum describing situations
        where EJB was found to be overkill and unnecessary. I am worried that
        selecting another technology could leave me doing major rework
        against a deployment deadline when I run in to the eventual
        performance bottleneck. What success have other Web developers had in
        passing performance limits?
        Keith Richardson
      • Dossy
        ... This does sound like a big cop-out. Can we not apply Yesterday s Weather to this problem? Do you have ANY customers? Or are you still in the haven t
        Message 3 of 22 , Jan 1, 2001
        • 0 Attachment
          On 2001.01.02, Keith Richardson <keith@...> wrote:
          > I think I am faced with having to take a copout of some form. We will
          > be creating solutions that are installed by our clients and used by
          > their customers. Some clients are large with jobs running through
          > hundreds of plants that are accessed though a single Web site. I have
          > absolutely no idea how often their customers will be accessing the
          > site - the clients don't know either. Trying to put a number of this
          > would be a wild guess at best.

          This does sound like a big cop-out. Can we not apply Yesterday's
          Weather to this problem?

          Do you have ANY customers? Or are you still in the "haven't sold
          product" stage and we're just doing a lot of what-if? (If your
          answers are "no" and "yes", then I'm tempted to say YAGNI. ;-) )

          If you already have a customer, can you not use them as a "model"
          for other customers in their size/scale/class, at least to create
          estimates that are even slightly better than WAGs?

          Can you not use measurements from smaller customers to approximate
          what measurements might look like for larger customers? Can you
          not create a spike to at least verify that these approximations
          are "close enough"? (Complexologists would suggest building
          simulations - I tend to agree.)

          It might be very costly to build a worthwhile simulation of a
          theoretical "huge" customer, but with the kind of dollars on
          the line for that kind of customer, wouldn't it be worth it?
          What would be more costly, the simulation or the loss of the
          customer because you didn't fully understand your own product
          that you're selling?

          > The worst that can happen if the most
          > scalable technology is selected is the project takes longer to
          > implement and the deployment costs are slightly higher. The worst
          > that can happen if we go for lightweight solutions is that we find
          > the system is unusable when it eventually gets deployed. That would
          > be bad!

          If you're really doing XP, the cost of change curve should be
          "flat enough" such that moving from a lightweight solution to a
          "scalable technology" shouldn't have an oppressive cost.

          If you implement the lightweight solution poorly, sure, it'll
          be that much tougher to enhance the system to meet scalability
          needs later on. But that's not a technology selection problem...

          Make the system usable when it's deployed, and make it deployable
          ASAP for the customers you DO know about. When a big fat customer
          comes along, the money they should pay should justify any serious
          enhancements to the system, and as long as you've kept the system
          as simple as possible, enhancing where it's necessary (and only
          where it's necessary) shouldn't be too hard.


          - Dossy

          --
          Dossy Shiobara mail: dossy@...
          Panoptic Computer Network web: http://www.panoptic.com/
        • Ron Jeffries
          ... With all due respect, I believe the above to be almost exactly incorrect. The worst that can happen if a big technology is chosen is that you get to market
          Message 4 of 22 , Jan 1, 2001
          • 0 Attachment
            At 02:55 AM 1/2/2001 +0000, it seemed like Keith Richardson wrote:
            >The worst that can happen if the most
            >scalable technology is selected is the project takes longer to
            >implement and the deployment costs are slightly higher. The worst
            >that can happen if we go for lightweight solutions is that we find
            >the system is unusable when it eventually gets deployed. That would
            >be bad!

            With all due respect, I believe the above to be almost exactly incorrect.

            The worst that can happen if a big technology is chosen is that you get to
            market late, lose market share, and don't learn what people really want
            until it is too late. The worst that can happen if you go for lightweight
            solutions is that you have to beef it up upon deployment. But you find that
            out about a week after you start.

            Flexibility is better than planning, every time.

            Ronald E Jeffries
            http://www.XProgramming.com
            http://www.objectmentor.com
          • Glen B. Alleman
            Flexibility versus planning? ... critical (realtime shop floor control) or is it informational to a non-critical set of users? That is a big range of
            Message 5 of 22 , Jan 1, 2001
            • 0 Attachment
              Flexibility versus planning?

              Here's some questions that may help clarify the situation:

              >>What is the context of the problem here?
              >>Where are the boundaries of your recommendation here?
              >>Where in the overall scheme of things does this system fit? Is it mission
              critical (realtime shop floor control) or is it informational to a
              non-critical set of users? That is a big range of "domain."

              The problem described by the original poster was a bit vague I agree. The
              manufacturing domain problems of "scalability estimates" are common,
              serious, and at the same time unknown without a lot of leg work. In a large
              manufacturing systems, "massive customization" is supported by a
              "configurator" based BOM system. See http://www.configsc.com/logia2.htm as
              an example of such a system. This is not the best, there are others (I2
              being a better one). This process can whipsaw the transaction rates by 2 to
              3 orders of magnitude when a simple "build to order" configuration is
              changed by marketing in an attempt to reposition a product line. The sizing
              impacts on this type of plant or shop floor system are a challenge for the
              experts, let along someone just getting started in the development of a
              plant data management system.

              This fellow needs to perform some analysis to determine the boundaries of
              the problem before embarking on ANY development method. He needs to get a
              strategy of how to scale the system "if and when" scaling is needed. If it
              is not needed, fine, but if it is then the system must be capable of
              performing this scaling without disruption to the ongoing business.

              If it is an on going business, then there are some CRUD stats somewhere,
              even if they have to be gathered by hand. Those then are used to "size" the
              solutions and determine the underlying technology needs. This is really
              simple IT Strategy stuff. There is definitely missing information in this
              case, which will most likely cause problems down stream. He's in no position
              to pick any technology without first understanding the processes and the
              data that they touch.

              Glen B Alleman
              Niwot Ridge Consulting

              "Every even number greater than 2 is the sum of two primes"


              >-----Original Message-----
              >From: Ron Jeffries [mailto:ronjeffries@...]
              >Sent: Monday, January 01, 2001 8:28 PM
              >To: extremeprogramming@egroups.com
              >Subject: Re: [XP] Re: Architecture for XP and scalable web sites
              >
              >
              >At 02:55 AM 1/2/2001 +0000, it seemed like Keith Richardson wrote:
              >>The worst that can happen if the most
              >>scalable technology is selected is the project takes longer to
              >>implement and the deployment costs are slightly higher. The worst
              >>that can happen if we go for lightweight solutions is that we find
              >>the system is unusable when it eventually gets deployed. That would
              >>be bad!
              >
              >With all due respect, I believe the above to be almost exactly incorrect.
              >
              >The worst that can happen if a big technology is chosen is that you get to
              >market late, lose market share, and don't learn what people really want
              >until it is too late. The worst that can happen if you go for lightweight
              >solutions is that you have to beef it up upon deployment. But you
              >find that
              >out about a week after you start.
              >
              >Flexibility is better than planning, every time.
              >
              >Ronald E Jeffries
              >
            • Patrick Logan
              ... I would notice that this story is ambiguous.
              Message 6 of 22 , Jan 1, 2001
              • 0 Attachment
                --- "Keith Richardson" <keith@s...> wrote:

                > How would you have handled the user story: "Must be able to
                > scale quickly to unpredictable levels when a new client wants
                > to deploy in a high load environment"?

                I would notice that this story is ambiguous.
              • Laurent Bossavit
                ... Wouldn t it be worse if you selected the most scalable technology, took longer to implement the project, *and* found the system didn t respond well to high
                Message 7 of 22 , Jan 2, 2001
                • 0 Attachment
                  > The worst that can happen if the most scalable technology is
                  > selected is the project takes longer to implement and the
                  > deployment costs are slightly higher. The worst that can happen if
                  > we go for lightweight solutions is that we find the system is
                  > unusable when it eventually gets deployed. That would be bad!

                  Wouldn't it be worse if you selected the most scalable technology,
                  took longer to implement the project, *and* found the system didn't
                  respond well to high loads when deployed ?

                  Do you have any guarantee that won't happen ? What steps can be
                  taken to prevent this happening ?


                  ========================================
                  We aim to make simple things simple and
                  complex things possible.
                  ========================================
                  Laurent Bossavit - Software Architect
                  >>> laurent.bossavit@... <<<
                  >>> 06 68 15 11 44 <<<
                  >> ICQ#39281367 <<
                  Agence Bless http://www.agencebless.com/
                  ========================================
                • Keith Richardson
                  ... What a great forum - I posted a message at 10pm and got many great responses by 7am!! Either you guys never sleep or most of you are not is the EST time
                  Message 8 of 22 , Jan 2, 2001
                  • 0 Attachment
                    --- In extremeprogramming@egroups.com, "Laurent Bossavit"
                    <laurent.bossavit@a...> wrote:
                    > > The worst that can happen if the most scalable technology is
                    > > selected is the project takes longer to implement and the
                    > > deployment costs are slightly higher. The worst that can happen if
                    > > we go for lightweight solutions is that we find the system is
                    > > unusable when it eventually gets deployed. That would be bad!
                    >
                    > Wouldn't it be worse if you selected the most scalable technology,
                    > took longer to implement the project, *and* found the system didn't
                    > respond well to high loads when deployed ?
                    >
                    > Do you have any guarantee that won't happen ? What steps can be
                    > taken to prevent this happening ?

                    What a great forum - I posted a message at 10pm and got many great
                    responses by 7am!! Either you guys never sleep or most of you are not
                    is the EST time zone!

                    The clear consensus is to keep the technology simple. Now to pick a
                    robust and simple technology that works well with XP. Java is a much
                    cleaner environment than ActiveX which would suggest not using MS-
                    based development environment for IIS. A new question: what
                    combination of development tools and deployment environment is best
                    for a Web application built using XP? It should have a clean object
                    model and a build environment that allows for rapid development cycle
                    times. It must also be robust and a good performer because I don't
                    want to be kludging around limitations or speed bottlenecks. What is
                    being used for XP and how is it working?
                    Keith Richardson
                  • Chad Fowler
                    ... cycle ... I would suggest that keeping your technology choice as open as possible is a good plan. This enables you to start with simplicity and grow when
                    Message 9 of 22 , Jan 2, 2001
                    • 0 Attachment
                      >
                      > The clear consensus is to keep the technology simple. Now to pick a
                      > robust and simple technology that works well with XP. Java is a much
                      > cleaner environment than ActiveX which would suggest not using MS-
                      > based development environment for IIS. A new question: what
                      > combination of development tools and deployment environment is best
                      > for a Web application built using XP? It should have a clean object
                      > model and a build environment that allows for rapid development
                      cycle
                      > times. It must also be robust and a good performer because I don't
                      > want to be kludging around limitations or speed bottlenecks. What is
                      > being used for XP and how is it working?
                      > Keith Richardson

                      I would suggest that keeping your technology choice as open as
                      possible is a good plan. This enables you to start with simplicity
                      and grow when the need arises with less investment than a proprietary
                      alternative would afford. We are successfully using and deploying
                      Enhydra (http://www.enhydra.org) in a change-heavy iterative
                      development environment. We have been very happy with its performance
                      and flexibility (as well as its price--Open Source).
                    • Stefan Schmiedl
                      They work 40 hour weeks and hence have lots of free time on their hands.... :-) Stefan +---------+------------------------- ...
                      Message 10 of 22 , Jan 2, 2001
                      • 0 Attachment
                        They work 40 hour weeks and hence have lots of free time on their
                        hands.... :-)

                        Stefan

                        +---------+-------------------------
                        | from | Keith Richardson <keith@...>
                        | to | extremeprogramming@egroups.com <extremeprogramming@egroups.com>
                        | date | 02.01.2001 16:13
                        | subject | [XP] Re: Architecture for XP and scalable web sites
                        +---------+-------------------------

                        K> What a great forum - I posted a message at 10pm and got many great
                        K> responses by 7am!! Either you guys never sleep or most of you are not
                        K> is the EST time zone!
                      • astl@home.com
                        ... using ... the ... be ... to ... application ... by environment ... no ... solution ... traffic. ... features ... Do the user stories say don t lose my
                        Message 11 of 22 , Jan 12, 2001
                        • 0 Attachment
                          --- In extremeprogramming@egroups.com, "Chad Fowler"
                          <chadfowler@y...> wrote:
                          > --- In extremeprogramming@egroups.com, "Keith Richardson"
                          > <keith@s...> wrote:
                          > > Hello,
                          > > This forum has included several messages describing problems
                          using
                          > XP
                          > > with EJB, describing EJB as non-OO and other difficulties with
                          the
                          > > EJB architecture. A highly scalable web site must be able to
                          > automate
                          > > persistance of session and environment data, allow deployment to
                          be
                          > > easily adjusted and many other features that EJBs (and more
                          > > completely J2EE) provides. Are there other environments that have
                          > > proven to be better for developing highly scalable solutions with
                          > XP
                          > > or should I interpret these messages as saying that XP is not
                          > > applicable for these needs? Any XP success stories in this area
                          to
                          > be
                          > > shared?
                          > > Keith Richardson
                          >
                          > I think the point of a lot of these messages is that EJB is not, at
                          > least at the beginning of a project, (in most cases) the simplest
                          > thing that could possibly work. A well factored servlet
                          application
                          > provides session persistence (not sure what you mean
                          by "environment"
                          > in this context) and (being well factored) would also give
                          > flexibility for deployment options (things are logically decoupled,
                          > so they can be physically separated down the road).
                          >
                          > In the past, our developers were assuming that scalable-web-
                          > application == J2EE-all-the-way and just going with EJB from the
                          > start. This instroduced complexity in the code, the
                          > development/build process, and the application server deployment.
                          > This complexity invariably lead to various problems (more points of
                          > potential failure).
                          >
                          > The approach we've taken semi-recently with our developers is to
                          > say, "You are free to use EJB (and the rest of the J2EE baggage) if
                          > you can explain why you need it." What we've found so far is that
                          no
                          > one has ever needed it, and we haven't had any related problems.
                          > (And, our applications are *so* much easier to deploy and maintain
                          > now!) In the mean time, we've been trying to keep things well
                          > factored, so they could be easily moved to the more complex
                          solution
                          > should we ever hit the scalability barriers of this approach. My
                          > guess is that we won't ever get there.
                          >
                          > Of course, we're no amazon.com, but we get some pretty heavy
                          traffic.
                          >
                          > Just as an addendum, here's my XP zealot answer:
                          > "must be able to automate persistance of session and environment
                          > data, allow deployment to be easily adjusted and many other
                          features
                          > that EJBs (and more completely J2EE) provides" wasn't in my user
                          > stories. :)
                          >
                          > Chad

                          Do the user stories say "don't lose my data that I am entering just
                          because you want to update the servers?"

                          Ken.
                        • Alan Francis
                          ... ... Can we PLEASE try to keep the copied text to a minimum ? Even if the two new lines are the most
                          Message 12 of 22 , Jan 12, 2001
                          • 0 Attachment
                            --- In extremeprogramming@egroups.com, astl@h... wrote:
                            <a zillion lines of copied message>
                            <followed by....>
                            > Do the user stories say "don't lose my data that I am entering just
                            > because you want to update the servers?"

                            Can we PLEASE try to keep the copied text to a minimum ? Even if the
                            two new lines are the most insightful piece of observation yet, it's
                            still anooying to get massive emails full of copied stuff.

                            Maybe OMI could host an extremeprogramming wiki so conversations
                            could be conversations ? :-)

                            Alan
                          • Chad Fowler
                            ... If they do, then it s time to find the simplest solution that can satisfy the story--again, not limited to EJB (nor very likely to be).
                            Message 13 of 22 , Jan 13, 2001
                            • 0 Attachment
                              > > Just as an addendum, here's my XP zealot answer:
                              > > "must be able to automate persistance of session and environment
                              > > data, allow deployment to be easily adjusted and many other
                              > features
                              > > that EJBs (and more completely J2EE) provides" wasn't in my user
                              > > stories. :)
                              > >
                              > Do the user stories say "don't lose my data that I am entering just
                              > because you want to update the servers?"
                              >
                              > Ken.

                              If they do, then it's time to find the simplest solution that can
                              satisfy the story--again, not limited to EJB (nor very likely to be).
                            • Erik Meade
                              I mentioned this to Ward a few weeks ago, he said that Extreme Programming is on topic for his (The) Wiki http://www.c2.com/cgi/wiki?FrontPage -- Erik Meade
                              Message 14 of 22 , Jan 17, 2001
                              • 0 Attachment
                                I mentioned this to Ward a few weeks ago, he said that Extreme
                                Programming is on topic for his (The) Wiki
                                http://www.c2.com/cgi/wiki?FrontPage

                                --
                                Erik Meade emeade@...
                                Senior Consultant Object Mentor, Inc.
                                http://www.junit.org


                                -----Original Message-----
                                From: "Alan Francis" <alan@...>
                                Subject: Re: Architecture for XP and scalable web sites

                                Maybe OMI could host an extremeprogramming wiki so conversations
                                could be conversations ? :-)

                                Alan
                              Your message has been successfully submitted and would be delivered to recipients shortly.