Loading ...
Sorry, an error occurred while loading the content.

Re: [agileDatabases] ANN: The Skillset of an Agile DBA

Expand Messages
  • Dawn Wolthuis
    ... I suspect you are correct that most would agree with that statement. I don t. I recognized that in my experience (strictly anecdotal), I spent far fewer
    Message 1 of 18 , Feb 23, 2006
    • 0 Attachment
      On 2/22/06, Curt Sampson <yahoo@...> wrote:
      > On Tue, 21 Feb 2006, Dawn Wolthuis wrote:
      >
      > > I think that it is time to introduce lists as attribute values
      > > back into the mix, for example, for much more agile modeling and
      > > implementation.
      >
      > This seems to me typical of where people have gone wrong in the agile
      > vs. database battle.
      >
      > I would suspect that most of us would agree with the following statements:
      >
      > a) The ability to rapidly and drastically change code (I like to
      > call this "rototilling") and deploy those changes, yet do so safely,
      > is wonderful and leads to faster and better software development.
      >
      > b) The relational model, by qualitatively simplifying the structure
      > of databases, freeing one from the bounds of having to chose data
      > access paths, and separating physical and logical concerns, is good
      > thing.

      I suspect you are correct that most would agree with that statement.
      I don't. I recognized that in my experience (strictly anecdotal), I
      spent far fewer dollars getting teams working with a non-RDBMS product
      to accomplish more than with SQL-DBMS (or IMS) tools. So a couple of
      years ago I started studying why this might be - was it just my
      experience or was there some meat to this?

      Obviously there is nothing obviously wrong with relational theory and
      it is mathematically fine, but I think there is something about
      applying that model for typical software development that makes it
      less agile than if you employ other models. I don't know if you
      looked at my blog, but instead of reproducing the same information
      here, I'll just suggest you might be interested in
      www.tincat-group.com/mewsings . There is no slam dunk case, but I
      think there is an excellent case for the industry to move away from
      use of the relational model for interfacing with databases.

      > c) Current implementations of the relational model, while falling
      > far short of the ideal, do indeed give us some of the advantages of
      > the model, though far from all of them.

      There are pros and cons to any solution, but there have been few
      agility improvements with databases such as Oracle over products that
      came before. Can you think of any way in which an Oracle is more
      agile for developers than Cache' (Intersystems) or U2 (IBM), for
      example? I can think of some other advantages, but agility doesn't
      make the list.

      > Our current DBMSes certainly make us suffer when we try to be agile.
      > It's a problem we need to attack.

      Yes, that is a focus for my work right now.

      > But I argue that this is far from a new problem, and previous solutions
      > did not follow the currently proposed method of attack.

      I often work with users of a very old solution that has many failings
      itself, but permits very agile software development and maintenance
      (PICK, aka, MultiValue). While I would suggest there are issues with
      any solution, I would like to see how we can take this more agile data
      model and improve it with what works well in relational databases
      without losing the agility or, possibly (although I'm less hopeful
      about this) add in those aspects that make it agile into existing
      RDBMS's. Most old data models, such as Berkeley DB (name/value
      pairs), U2 (PICK), and Cache' (MUMPS), are hosted by products that
      look their age. But in spite of that, they have some significant
      advantages over SQL-DBMS tools.

      > Procedural languages such as C make it very hard to be agile. One can
      > barely do unit testing, much less write a DSL, and as soon as one starts
      > to do something complex, the syntax barriers are formidable. However,
      > we didn't solve this by continuing to use C; we created other languages
      > that put fewer barriers in our way.

      I'm sure there are many who will continue to use SQL til the cows come
      home, but I agree that other languages are in order. My opinion is
      that these other languages should implement data models other than the
      relational model (XQuery would be an example), while others think that
      Tutorial-D (better relational model language than SQL) is in order.

      > In the same way, it's not at all the relational model itself that is
      > interfering with our ability to be agile in the DBMS world;

      Yes, it is. As I mention in my writing this week, the RM is not
      sufficient for software development. That means we necessarily must
      cross a chasm, even if sometimes small, to use it. Remove that gap
      and you gain agility.

      > it's the
      > current DBMS products.

      those too

      > Why is it so hard to concisely specify several
      > different interfaces to a set of base data? Why is it so hard to do even
      > simple renaming refactorings? Why can't we write our queries in a much,
      > much more modular fashion? None of this has anything to do with the
      > relational model; it has to do with vendors giving us products that are,
      > from the agile point of view, crap.

      Yes, it is related to the relational model in that the model has to do
      with the representation (that is Codd's claim with which I agree) of
      data structures with relations as the only first class citizen. If
      you were to put another interface to the data and permit lists in it,
      for example, it would cease to be the relational model. The RM is not
      about how data are stored (in theory) but how they are represented in
      the interface between the database and the world.

      > So rather than dumping the good stuff because we're getting bad stuff
      > along with it, we should be trying to get rid of the bad stuff.

      That was my original thinking until I did further research. I now
      believe we need to leave the RM behind, possibly by enhancing it so it
      is no longer the RM, but it might be preferable to start with other
      successful data models and improve the products employing those. They
      really do seem to lead to significant gains in agility.

      > Ideally, I thike we should be moving the relational model into our OO
      > programs, as well.

      There are many people who think that. I think that is the opposite of
      what needs to happen. Because it is impossible to create a view that
      meets the requirements of the RM (the Information Principle) and have
      that as a single view of the data for a UI (which I recognize was not
      the purpose of the RM) you simply cannot employ the RM head to toe in
      software development. You must switch models at some point to get
      such constructs as ordered lists. I think I say this better in my
      blog writing, but hopefully this gives an idea.

      > OO introduced one level of abstraction in data access by allowing us to
      > substitute similar storage and search algorithms for each other. For
      > example, in Java, when I want to read the second element stored in a
      > List object, I have no need to care about whether it's an ArrayList or a
      > LinkedList. I can delegate the responsiblity for deciding the specific
      > storage layout and access methods to the most appropriate place in my
      > program, and I can change it at will.

      Yes. And you cannot employ such abstractions and retain the
      relational model. The RM was seeming slimy to me in that whenever I
      tried to capture what it was, someone said it wasn't that. That is
      one of the reasons I decided to do more research. Nailing down
      precisely what the RM is might lead you, as it did me, to see that we
      need to move beyond it.

      > OO should move to the next level of abstration, the relational one, by
      > extending this abstraction to more complex searches. Given a collection
      > of items, I ought to be able to say, "give me all of the ones where the
      > e-mail address ends in @...", without having to worry about
      > how the data are stored or how the search will be done.

      Yes! But that is not the RM that gives you that. You can get that in
      other languages too (such as XQuery, although I'm not as much an
      XML/XQuery proponent as I might sound) without having to reduce all of
      your data structures to relations. You can work with a "web"
      (di-graph with tress on the nodes) of data.

      > Being able to
      > delegate that kind of thing to the most appropriate part of the program
      > gives me further agility.
      >
      > Think about how much time you spend constructing hash tables and pointer
      > chains, and trying to remember just how things are stored and indexed.
      > That could all go away, if the language and/or libraries were more
      > powerful.

      I agree with this last statement and my goals are similar to yours. I
      simply think that we, as a profession, need to ditch the RM in order
      to move further faster. Cheers! --dawn

      --
      Dawn M. Wolthuis
      Tincat Group, Inc.
      www.tincat-group.com

      Take and give some delight today!
    Your message has been successfully submitted and would be delivered to recipients shortly.