Loading ...
Sorry, an error occurred while loading the content.

Re: [agileDatabases] On Relational Theory

Expand Messages
  • Adrian Mowat
    Scott, Despite the storm of protest you have endured, I think I can see what you are getting at and I agree up to a point. I come from an ETL background
    Message 1 of 13 , Apr 19, 2006
    • 0 Attachment
      Scott,
       
      Despite the storm of protest you have endured, I think I can see what you are getting at and I agree up to a point.
       
      I come from an ETL background (moving data around between loosly related systems using a specialist tool) rather than a pure RDBMS background.  As a result, the structure of the data in my source and target systems has a huge impact on the design of the ETL process I build to move data between them.  I think relational theory is a very important aspect of system design and used well it can produce elegant, useful and maintainable systems.  On the other hand, if the design is driven entirely by theory over how the system will be used, it will not be very good.
       
      Take star schemas for example.  They are an elegant application of relational theory and the uninitiated may decide they can be used everywhere.  However, when you understand that a star schema is optimised for data mining operations and might be slow to populate and/or query, then you might re-consider your view and take a different approach for your performance critical data address repository.  In fact, you might not even want normalise to the 3rd Form.
       
      My point is that relational theory is valid, proven and useful - but it's more like a tool or langauge that can be applied well or badly depending on who is using it.
       
      I hope this is in line with your thinking, but I would be very interested to hear your thoughts
       
      Cheers
       
      Adrian
       
       


       
      On 18/04/06, Rakesh Patel <rakesh.mailgroups@...> wrote:
      Scott,

      this is quite a silly mail to send out.

      Let me ask you this, because this is what i think it boils down to:

      Would you rather work on a database schema designed by someone who knew
      relational theory or someone who did not?

      As someone who has had to try and decipher schemas where the column is
      just called status with no referential intergity constraints AT ALL in
      the db and a dozen or more possible tables that it could come from, I'd
      take the relational theorist anyday.

      Cheers

      Rakesh


      Scott W. Ambler wrote:

      >Have you ever been accosted, either at work or online, by some
      >"theory knob" who tried to convince you of the importance of
      >relational theory, and then when he wasn't satisfied with your answer
      >(hint: there was never a chance that he would be), he then claimed
      >that whatever new-fangled technique you were discussing was clearly
      >rubbish?  If so, you might find "On Relational Theory" at
      >http://www.agiledata.org/essays/relationalTheory.html to be an
      >interesting read.  Relational theory has its place, don't get me
      >wrong, but it's nowhere near as important as the theory knobs would
      >lead us to believe.
      >
      > From what I've seen over the years, relational theory is important when:
      >1. You're developing a relational database engine.
      >2. You're a computer science academic.
      >3. That's all you know, or at least that's what you prefer to focus on.
      >4. You're focused on past glories.
      >5. You can't find anything better to talk about over a couple of beers.
      >
      >So why do people ask about relational theory?  From what I've seen,
      >there are several reasons:
      >1. That's all they know, or at least that's what they focus on.
      >2. They're looking for an excuse not to change.
      >3. They're a zealot.
      >4. They think that it drives development efforts.
      >5. They've been misguided by the one-trick ponies, the fearful, and
      >the zealots among us.
      >
      >Why is relational theory an issue for someone who is clearly a
      >practitioner?  Because of the damage within the IT industry that I'm
      >seeing caused in its name, I've become concerned. As I noted, some
      >people use relational theory as an excuse not to change, but frankly
      >that's their business and I'm happy to let them travel along their
      >own path.  But other people, in particular college instructors and
      >book writers, needlessly inflict relational theory on people who are
      >trying to learn how to become an effective data professional, or
      >better yet an effective IT professional.  Too much focus on theory
      >can really make data-oriented development techniques unattractive to
      >practitioners, which is one of the reasons why I think so many
      >application developers seem to have little or no skills in this area.
      >
      >Anyway, the article goes into more detail on these ideas.  Any
      >feedback that you have would be appreciated.
      >
      >- Scott
      >
      >====================================================
      >Scott W. Ambler  :-)
      >Senior Consultant, Ambysoft Inc.
      >www.ambysoft.com/scottAmbler.html
      >
      >Refactoring Databases: Evolutionary Database Design
      >(www.ambysoft.com/books/refactoringDatabases.html ) is now available!
      >
      >
      >
      >
      >Yahoo! Groups Links
      >
      >
      >
      >
      >
      >
      >
      >

      >



    • Dawn Wolthuis
      While I can appreciate theory and really do care that it aligns with practice, perhaps Scott is indicating that because knowing relational theory has so little
      Message 2 of 13 , Apr 20, 2006
      • 0 Attachment
        While I can appreciate theory and really do care that it aligns with practice, perhaps Scott is indicating that because knowing relational theory has so little to do with being excellent in data-based software development, it is far more important that someone knows how to design and implement solutions that work for today and are flexible for tomorrow.  It is a shame that in higher ed considerable time is often spent teaching relational theory, with little time for best practices.  While I would suggest that theory is a good foundation but that relational theory is inadequate, Scott is more inclined to take the products that are out there and work with them (I want them to ditch the RM) and develop best practices.  So we both have some healthy disrespect for relational theory, but we come from different perspectives to that point.

        I disagree that relational theory ensures accuracy of data.  You can find just as much inaccurate data in systems with constraints specified to the dbms as those where they are coded in programming languages, for example, in spite of seemingly sound theory that suggests otherwise.  The issues you find vary, with issues in SQL-DBMS's often including vast amounts of data missing or oversimplified in order to accomodate constraints or simplify the data model for first normal form.  Developers decide not to take in multiple values for an attribute simply because that would lead to another table.   Sure, it does help ensure that data that does not fit a particular set of constraints can get into the database, but the cleverness that both end-users and application developers employ to contend with no-longer-accurate constraints leads to bad data as well.    --dawn

        --
        Dawn M. Wolthuis
        Tincat Group, Inc.
        www.tincat-group.com

        Take and give some delight today!

        On 4/19/06, Golferd, Ilay (NIH/NIAID) [C] <igolferd@...> wrote:
        relational theory ( that is if people understand it and know how to apply) ensures accuracy of data without relaying on the Bug-less application code( which does not exist).


        From: Scott W. Ambler [mailto:swa@...]
        Sent: Thursday, April 13, 2006 8:00 AM
        To: agileDatabases@yahoogroups.com

        Subject: [agileDatabases] On Relational Theory

        Have you ever been accosted, either at work or online, by some
        "theory knob" who tried to convince you of the importance of
        relational theory, and then when he wasn't satisfied with your answer
        (hint: there was never a chance that he would be), he then claimed
        that whatever new-fangled technique you were discussing was clearly
        rubbish?  If so, you might find "On Relational Theory" at
        http://www.agiledata.org/essays/relationalTheory.html to be an
        interesting read.  Relational theory has its place, don't get me
        wrong, but it's nowhere near as important as the theory knobs would
        lead us to believe.

        From what I've seen over the years, relational theory is important when:
        1. You're developing a relational database engine.
        2. You're a computer science academic.
        3. That's all you know, or at least that's what you prefer to focus on.
        4. You're focused on past glories.
        5. You can't find anything better to talk about over a couple of beers.

        So why do people ask about relational theory?  From what I've seen,
        there are several reasons:
        1. That's all they know, or at least that's what they focus on.
        2. They're looking for an excuse not to change.
        3. They're a zealot.
        4. They think that it drives development efforts.
        5. They've been misguided by the one-trick ponies, the fearful, and
        the zealots among us.

        Why is relational theory an issue for someone who is clearly a
        practitioner?  Because of the damage within the IT industry that I'm
        seeing caused in its name, I've become concerned. As I noted, some
        people use relational theory as an excuse not to change, but frankly
        that's their business and I'm happy to let them travel along their
        own path.  But other people, in particular college instructors and
        book writers, needlessly inflict relational theory on people who are
        trying to learn how to become an effective data professional, or
        better yet an effective IT professional.  Too much focus on theory
        can really make data-oriented development techniques unattractive to
        practitioners, which is one of the reasons why I think so many
        application developers seem to have little or no skills in this area.

        Anyway, the article goes into more detail on these ideas.  Any
        feedback that you have would be appreciated.

        - Scott

        ====================================================
        Scott W. Ambler  :-)
        Senior Consultant, Ambysoft Inc.
        www.ambysoft.com/scottAmbler.html

        Refactoring Databases: Evolutionary Database Design
        (www.ambysoft.com/books/refactoringDatabases.html) is now available!





        YAHOO! GROUPS LINKS





      • Scott W. Ambler
        Answering several at once, sorry for any confusion. - Scott ... No. And, when was the last time you heard someone say the person doesn t know OO theory,
        Message 3 of 13 , Apr 20, 2006
        • 0 Attachment
          Answering several at once, sorry for any confusion.

          - Scott


          On Wed, April 19, 2006 9:41 am, Charles Abreu said:
          > Isn't this the same as saying "don't bother if she don't know OOP as soon
          > as
          > she can draw the adequate UML diagrams"

          No. And, when was the last time you heard someone say "the person doesn't
          know OO theory, therefore they don't know what they're doing"?

          I've been very clear in the past that you need to understand the
          technology in order to design for it effectively. I'm sure you know many
          good OO programmers who don't have a background in OO theory. I'm also
          sure that you know good OO programmers who do have such a background.
          It's the same thing on the relational side.

          >
          > I really think for any practice the foundation theory is aways a
          > requisite.

          There seems to be evidence to the contrary. Sure, the good developers
          among us without the theory background have likely figured out some of the
          fundamentals themselves.

          > If someone says he is not dependent on theory one of two cases is true:
          > or
          > he is doing a mechanical job or he is applying the theory anyway even he
          > don't admit it.

          Could be. Yet either way, you don't seem him running around claiming how
          wonderful theory is and how everyone needs to have a solid background in
          it, do you?


          >
          > I really don't think you mean that theory is never requisite for any
          > practice. So, what are you trying to state is that relational theory isn't
          > the (most) adequate foundation for relational databases practices? Hope
          > I'm
          > not missing more options.

          Yes. Many people, for many years, have been pointing out that relational
          theory is inadequate for RDBMSs, let alone for working with them. We can
          also see this through the actions of the vendors, all of whom are
          motivated to provide valuable functionality to their customers. It's time
          that we recognice this.




          On Wed, April 19, 2006 10:04 am, Golferd, Ilay \(NIH/NIAID\) [C] said:
          > relational theory ( that is if people understand it and know how to
          > apply) ensures accuracy of data without relaying on the Bug-less
          > application code( which does not exist).
          >

          That's an interesting theory, but unfortunately the current state of
          production databases seem to show that this doesn't actually seem to be
          happening in practice. Seems to me that if you want to ensure accuracy of
          data, that perhaps you should be investing some effort in putting together
          a full regression test suite for your database. I personally don't
          remember any of the theory guys ever talking about this, although perhaps
          my memory isn't what it used to be. Anyway, I've shared some thoughts on
          this subject at www.agiledata.org/essays/databaseTesting.html


          On Wed, April 19, 2006 8:08 am, Adrian Mowat said:
          > Scott,
          >
          > Despite the storm of protest you have endured, I think I can see what you
          > are getting at and I agree up to a point.
          >
          > I come from an ETL background (moving data around between loosly
          > related systems using a specialist tool) rather than a pure RDBMS
          > background. As a result, the structure of the data in my source and
          > target
          > systems has a huge impact on the design of the ETL process I build to move
          > data between them. I think relational theory is a very important aspect
          > of
          > system design and used well it can produce elegant, useful and
          > maintainable
          > systems. On the other hand, if the design is driven entirely by theory
          > over
          > how the system will be used, it will not be very good.

          Exactly. Theory can help you to come to a good design, but a purely
          theoretical design is very likely going to be a disaster. You need to
          have hands-on experience with the technologies that you're designing for,
          and you need to be prepared to take an evolutionary approach because
          you'll discover that no matter how good you are you'll never quite get it
          right.


          >
          > Take star schemas for example. They are an elegant application of
          > relational theory and the uninitiated may decide they can be
          > used everywhere. However, when you understand that a star schema is
          > optimised for data mining operations and might be slow to populate and/or
          > query, then you might re-consider your view and take a different approach
          > for your performance critical data address repository. In fact, you might
          > not even want normalise to the 3rd Form.

          Exactly. Hands-on experience helps to bring you to that conclusion.

          It's interesting to observe that the most vociferous of the theory
          protagonists rarely seem to "get their hands dirty" actually building
          things.

          >
          > My point is that relational theory is valid, proven and useful - but it's
          > more like a tool or langauge that can be applied well or badly depending
          > on
          > who is using it.

          Yes, and my point is that many people can be very effective without that
          tool just like people can be effective with it.

          - Scott
        • Chris Holmes
          Yes, and my point is that many people can be very effective without that tool just like people can be effective with it. - Scott They can be effective, yes.
          Message 4 of 13 , Apr 20, 2006
          • 0 Attachment
            Yes, and my point is that many people can be very effective without that
            tool just like people can be effective with it.

            - Scott


            They can be effective, yes. But if they are effective then at some
            subconscious level they are applying relational theory. They may not be
            aware that what they are doing is based in relational theory, but if
            they're normalizing tables and designing for constraints, etc., then
            they they're applying theory.


            I played guitar for 13 years before I sat down and learned any formal
            "theory". When I would improvise during a jam session I stuck to certain
            notes and avoided others. Experience taught me which notes to avoid, but
            occasionally I would screw up. I never was consciously aware of what I
            was doing, but I was applying musical theory; staying within the
            confines of a given scale for a given key. I didn't know what key or
            what scale, I just knew them instinctively from years of application and
            playing.

            After I learned formal theory it opened my eyes (and ears) to whole news
            ways of thinking about music. It made me a better musician.

            I would never discourage someone from learning relational theory. I
            think it can only help them in the future to build better schemas. The
            real problem is when folks who learn theory become beholden to it at the
            expense of solving real problems. This is a common problem in the
            musical world as well; there are musicians who worship at the altar of
            theory and are completely unable to break out of the theoretical
            constraints they've placed on themselves. They turn their noses up at
            musicians who are able to do just that.
          • Scott W. Ambler
            ... This is exactly the issue that I see in the data world. - Scott ==================================================== Scott W. Ambler :-) Senior
            Message 5 of 13 , Apr 21, 2006
            • 0 Attachment
              At 03:09 PM 4/20/2006, Chris wrote:
              >I would never discourage someone from learning relational theory. I
              >think it can only help them in the future to build better schemas. The
              >real problem is when folks who learn theory become beholden to it at the
              >expense of solving real problems. This is a common problem in the
              >musical world as well; there are musicians who worship at the altar of
              >theory and are completely unable to break out of the theoretical
              >constraints they've placed on themselves. They turn their noses up at
              >musicians who are able to do just that.

              This is exactly the issue that I see in the data world.

              - Scott

              ====================================================
              Scott W. Ambler :-)
              Senior Consultant, Ambysoft Inc.
              www.ambysoft.com/scottAmbler.html

              Refactoring Databases: Evolutionary Database Design
              (www.ambysoft.com/books/refactoringDatabases.html) is now available!
            Your message has been successfully submitted and would be delivered to recipients shortly.