Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Re: Architecture for XP and scalable web sites

Expand Messages
  • Ron Jeffries
    At 05:27 PM 1/2/2001 +0100, it seemed like ... This makes me wonder whether you really wish you had an object database instead of relational. Can you tell us a
    Message 1 of 21 , Jan 2, 2001
    View Source
    • 0 Attachment
      At 05:27 PM 1/2/2001 +0100, it seemed like
      robert.claeson@... wrote:
      >The requirements for many of the applications that we develop often causes
      >the database schema to become quite complicated, on the order of several
      >hundred tables. Most database queries target multiple tables. CMP usually
      >just plain doesn't work satisfactorily under those conditions. If youonly
      >need to access one or two tables, then fine, go ahead and use CMP. That's
      >an utterly simple access, though.

      This makes me wonder whether you really wish you had an object database
      instead of relational. Can you tell us a little about what kind of app
      wants hundreds of tables and lots of multiple-table queries?

      Thanks!

      Ronald E Jeffries
      http://www.XProgramming.com
      http://www.objectmentor.com
    • Anders Bengtsson
      On Tue, 2 Jan 2001 robert.claeson@brightinteractive.com wrote:But why would you want your application to be anything but the simplest? The
      Message 2 of 21 , Jan 2, 2001
      View Source
      • 0 Attachment
        On Tue, 2 Jan 2001 robert.claeson@... wrote:

        > >But why would you want your application to be anything but the simplest?
        >
        > The requirements for many of the applications that we develop often causes
        > the database schema to become quite complicated, on the order of several
        > hundred tables. Most database queries target multiple tables. CMP usually
        > just plain doesn't work satisfactorily under those conditions. If youonly
        > need to access one or two tables, then fine, go ahead and use CMP. That's
        > an utterly simple access, though.

        We had just over 30 tables, with mostly one-way relations between them.
        There were, as you say, problems in the few cases where there were many
        tables involved.

        > I'm mostly refering to WebSphere here, but TopLink and WebLogic shows
        > similar performance profiles. There's also a product named "Object
        > Extender" for VA Smalltalk that's the predecessor to the CMP stuff in
        > WebSphere and VA Java. We've had tons of problems with that as well,
        > especially if persistent objects inherit from each other.

        I've heard bad things beeing said about the entity bean performance of
        WebSphere, but one of their architechts told me that they in fact had
        good performance. :-)

        > >That's true (assuming you have an RMI server that optimizes calls within
        > >the same JVM).
        >
        > I'm referring to the case where there are multiple servlet containers
        > accessing a shared RMI server through a network connection.

        If your app is really fast, compared to the load, its really nice to be
        able to run it all in a single JVM. Both the speed of running it all
        in one JVM and the comfort of knowing that you can buy yourself out of
        performance problems.

        > Fact is, getting acquainted with a good application server and using it
        > for both small and large applications helps both quality and productivity.
        > In this case, there are good reasons for not paying too much attention to
        > YAGNI. Too many web sites has been developed too "small", and has been
        > facing tremendeous performance problems when the load increases. YDKIYGNI
        > ("You Don't Know If You're Gonna Need It").

        This turns out to the biggest problem of all in building web applications,
        for me at least. You never really know how "big" your solution needs to
        be, and that single decision affects the entire project greatly.
        I guess the true XP way would be to adopt it as you go, to avoid that kind
        of decision. Some day I'll have that kind of courage. :)
        In the meantime I'll take your advice and find out the ins and outs of my
        app server.

        /Anders
        _____________________________________________________________________
        A n d e r s B e n g t s s o n ndrsbngtssn@...
        Stockholm, Sweden


        _________________________________________________________
        Do You Yahoo!?
        Get your free @... address at http://mail.yahoo.com
      • Anders Bengtsson
        On Tue, 2 Jan 2001, Ron Jeffries wrote:This makes me wonder whether you really wish you had an object database instead of relational. Can you tell us a
        Message 3 of 21 , Jan 2, 2001
        View Source
        • 0 Attachment
          On Tue, 2 Jan 2001, Ron Jeffries wrote:

          > This makes me wonder whether you really wish you had an object database
          > instead of relational. Can you tell us a little about what kind of app
          > wants hundreds of tables and lots of multiple-table queries?

          Who _don't_ wish that they had an object database? I've been wishing
          that for years! ;-)

          _____________________________________________________________________
          A n d e r s B e n g t s s o n ndrsbngtssn@...
          Stockholm, Sweden


          _________________________________________________________
          Do You Yahoo!?
          Get your free @... address at http://mail.yahoo.com
        • robert.claeson@brightinteractive.com
          ... causes ... You bet. I ve always wished that. However, there was none reasonably priced object database that had all the other features that we needed.
          Message 4 of 21 , Jan 2, 2001
          View Source
          • 0 Attachment
            Me:

            >The requirements for many of the applications that we develop often
            causes
            >the database schema to become quite complicated, on the order of several
            >hundred tables. Most database queries target multiple tables. CMP usually
            >just plain doesn't work satisfactorily under those conditions. If youonly
            >need to access one or two tables, then fine, go ahead and use CMP. That's
            >an utterly simple access, though.

            Ron Jeffries:

            >This makes me wonder whether you really wish you had an object database
            >instead of relational.

            You bet. I've always wished that. However, there was none reasonably
            priced object database that had all the other features that we needed.
            Nowadays, I try to find ways to use Oracle 8i or DB2 as much as possible.
            They have reasonably object-oriented features to make the mapping easier.

            >Can you tell us a little about what kind of app
            >wants hundreds of tables and lots of multiple-table queries?

            It's actually a quite boring application. It's an administrative
            application for member organizations. It manages various levels of
            membership, functions within the organization that members may have, local
            chapters, benefits, invoicing, accounts receivables, insurance policies,
            subscriptions and what not. All in all a pretty complete package for what
            many member organizations need. Most aspects are configurable. The first
            version was, incidentally, written in Smalltalk. It has since been
            rewritten in Java for various reasons.

            The invoicing run is, well, complicated. It took almost a day to run when
            we had implemented it using VisualAge's persistence stuff. We then tried
            to rewrite it using TopLink, which didn't improve things much. We finally
            ended up rewriting it as Oracle stored procedures in PL/SQL (mind you, we
            didn't have much prior experience from PL/SQL), which decreased the
            execution time to 1 minute and 58 seconds using the same data set (~1 GB
            of invoice detail data). It took two people just two days to do that
            rewrite, including debugging, which I guess shows the effectiveness of
            pair programming for solving problems like these.

            /Robert
          • C. Keith Ray
            [...] ... [...] For $699 you can try a spike with WebObjects.
            Message 5 of 21 , Jan 2, 2001
            View Source
            • 0 Attachment
              [...]
              > The clear consensus is to keep the technology simple. Now to pick a
              > robust and simple technology that works well with XP. Java is a much
              > cleaner environment than ActiveX which would suggest not using MS-
              > based development environment for IIS. A new question: what
              > combination of development tools and deployment environment is best
              > for a Web application built using XP? It should have a clean object
              > model and a build environment that allows for rapid development cycle
              > times. It must also be robust and a good performer because I don't
              > want to be kludging around limitations or speed bottlenecks. What is
              > being used for XP and how is it working?
              > Keith Richardson
              [...]

              For $699 you can try a spike with WebObjects.
              <http://www.apple.com/webobjects/techspecs.html>
            • Patrick Logan
              ... There are fairly simple star schemas published for these kinds of applications. Star schemas are a good fit for relational databases, are easy to
              Message 6 of 21 , Jan 3, 2001
              View Source
              • 0 Attachment
                --- robert.claeson@b... wrote:
                >
                > It's actually a quite boring application. It's an administrative
                > application for member organizations...
                > chapters, benefits, invoicing, accounts receivables,
                > insurance policies, subscriptions and what not...

                There are fairly simple star schemas published for these kinds
                of applications. Star schemas are a good fit for relational
                databases, are easy to understand, and happen to map well into
                object designs. Look for books by Kimble and/or Adamson &
                Venerable.

                > ...It took almost a day to run when
                > we had implemented it using VisualAge's persistence stuff.
                > ...using TopLink, which didn't improve things much. We finally
                > ended up rewriting it as Oracle stored procedures...
                > It took two people just two days to do that
                > rewrite, including debugging, which I guess shows the
                > effectiveness of pair programming...

                Frankly, I think it also shows how many problems do not need
                O/R mapping tools or an OODBMS.
              • Patrick Logan
                ... There are some practices that can simplify these schema, and so help you move faster in your XP project. One is not to use NULL in the database. Instead
                Message 7 of 21 , Jan 3, 2001
                View Source
                • 0 Attachment
                  --- Anders Bengtsson <ndrsbngtssn@y...> wrote:

                  > We had just over 30 tables, with mostly one-way relations
                  > between them. There were, as you say, problems in the few
                  > cases where there were many tables involved.

                  There are some practices that can simplify these schema, and
                  so help you move faster in your XP project.

                  One is not to use NULL in the database. Instead have a row
                  that represents the "null". If not all organizations have
                  a related whatsit, then instead of having their whatsit
                  column be NULL, have the whatsit column refer to a row in
                  the whatsit column that represents the "no whatsit" whatsit.

                  This is kind of like the Null Object pattern. (See Ward
                  Cunningham's Checks pattern language at www.c2.com)

                  Even better is to use this "null row" pattern along with
                  the star schema pattern (same web site as Checks).

                  The star schema pattern simpifies the relationships between
                  tables, making the schema easier to understand, more
                  "object/relational friendly", better performing, more
                  maintainable, and less prone to mistakes.

                  All good for XP projects.
                • Anders Bengtsson
                  On Wed, 3 Jan 2001, Patrick Logan wrote:One is not to use NULL in the database. Instead have a row that represents the null . If not all organizations
                  Message 8 of 21 , Jan 4, 2001
                  View Source
                  • 0 Attachment
                    On Wed, 3 Jan 2001, Patrick Logan wrote:

                    > One is not to use NULL in the database. Instead have a row
                    > that represents the "null". If not all organizations have
                    > a related whatsit, then instead of having their whatsit
                    > column be NULL, have the whatsit column refer to a row in
                    > the whatsit column that represents the "no whatsit" whatsit.
                    >
                    > This is kind of like the Null Object pattern. (See Ward
                    > Cunningham's Checks pattern language at www.c2.com)

                    That's interesting, I've never thought about using Null Object for
                    databases, even though I've used it in other contexts.

                    > Even better is to use this "null row" pattern along with
                    > the star schema pattern (same web site as Checks).
                    >
                    > The star schema pattern simpifies the relationships between
                    > tables, making the schema easier to understand, more
                    > "object/relational friendly", better performing, more
                    > maintainable, and less prone to mistakes.

                    The schema we used initially was simply whatever the automated persistence
                    of our EJB server chose to store the beans as. It actually worked great in
                    most cases, but I guess it was kind of asking for trouble, since it meant
                    that we didn't really spend any time thinking about schema design.
                    On the other hand, there weren't really many problems with it, so in the
                    end I guess it did actually save time.
                    So, even "TSTTCPW by accident" works some times. :)

                    /Anders
                    _____________________________________________________________________
                    A n d e r s B e n g t s s o n ndrsbngtssn@...
                    Stockholm, Sweden


                    _________________________________________________________
                    Do You Yahoo!?
                    Get your free @... address at http://mail.yahoo.com
                  Your message has been successfully submitted and would be delivered to recipients shortly.