Loading ...
Sorry, an error occurred while loading the content.

ANN: Agile Data Techniques

Expand Messages
  • Scott W. Ambler
    In December 2005 the Cutter Consortium published a collection of articles entitled Agile Data Techniques which I edited. A PDF can be downloaded free of
    Message 1 of 1 , Feb 24, 2006
      In December 2005 the Cutter Consortium published
      a collection of articles entitled "Agile Data
      Techniques" which I edited. A PDF can be
      downloaded free of charge from

      This publication contains 5 articles:
      1. Evolutionary Database Development Techniques
      by myself and Pramod J. Sadalage. This article
      summarizes Chapter 1 from our forthcoming
      Refactoring Databases
      which will be available in early March. It
      describes the fundamentals of database
      refactoring, a technique which enables you to
      safely improve the design of your database schema
      in small, evolutionary steps. It then describes
      how to take an agile approach to data modeling
      via an Agile Model Driven Development (AMDD)
      approach so that you think your database design
      through before you build it. Test-driven
      development (TDD) to concretely ensure data
      quality is described, as is the need to provide
      developers with their own sandboxes to work
      in. Finally, the article argues for a coherent
      approach to configuration management of
      data-related assets: if it’s important enough to
      create, it’s important enough to configuration manage.

      2. Agile Database Testing by Ken Collier. Collier
      describes his experiences taking an agile
      approach to data warehousing, focusing on how to
      go about testing your database
      effectively. Agile software developers are
      “quality infected”, they understand the need to
      provably develop high-quality, easy to maintain
      source code. Similarly, agile data professionals
      are also quality infected, understanding that the
      data sources which they develop and maintain must
      be of provably high quality too. The word
      “provably” is the crux of the
      problem. Traditional data professionals will go
      on for hours about the importance of data
      quality, and rightfully so, and argue for strict
      policies and procedures to ensure that
      quality. Experience, in the form of questionable
      production databases, shows that the traditional
      approach doesn’t work in practice. Instead, just
      as agile developers write a test before they
      write production code so must data
      professionals. Without a 100% regression test
      suite in place, which you can run at a moments
      notice, any claims of “data quality” are nothing
      more than hot air. Collier’s article presents
      workable strategies for testing data assets within your organization.

      3. Enabling Agile Database Development: A Case
      Study by Nick Ashley. Ashley describes his
      experiences as an agile DBA on a ThoughtWorks
      project for AOL in the UK. He describes how the
      development team had shared data-oriented
      activities amongst themselves for 18 months until
      they realized the need for some database
      expertise. Upon assessing the situation, the
      solution was to add actual value to the team by
      helping them to improve their database build,
      testing, and refactoring efforts (among other
      things). Interestingly, he chose to become part
      of the team, not an outsider trying to exert
      control over the team’s data assets.

      4. Agile Applications as Open-Vocabulary English
      Business Rules over SQL by Adrian Walker. Walker
      argues for an Agile Modeling (AM)-based approach
      where the stakeholders are actively involved in
      the development effort, and to support that you
      need inclusive techniques and tools which
      stakeholders understand. Instead of focusing on
      data modeling, what he describes as a brittle and
      slow strategy, Walker instead takes a business
      rules-based approach using a tool which generates
      both working software and the database schema to
      support it. The article is the most technical
      in this issue, showing the actual business rules
      for a supply-chain system and the SQL code
      generated to support them. You’ll be impressed
      by how easy the business rules are to understand
      – yes, stakeholders will still need a bit of help
      to write them – and how quickly you could develop
      a working system using this approach.

      5. Data and Agile Methods by David Hay. David
      Hay, a leading thinker in the data community and
      author of Data Model Patterns (1995) and
      Requirements Analysis (2002) makes the case for
      modeling in Data and Agile Methods. Modeling is
      a topic that is near and dear to my heart, but by
      arguing for the Zachman Framework he may discover
      that his approach is far too heavy for agile
      development. Agilists do in fact model,
      including Extreme Programmers (XPers), a fact
      which doesn’t come out well in this article: but
      we do so in a light-weight and streamlined
      manner. In the agile world just in time (JIT)
      model storming is key to your success – if you
      can model something today you clearly must have
      the ability to model it JIT when you actually
      need the information. By modeling JIT you’re
      more effective than modeling up front because the
      later you wait the better an understanding of the
      problem domain you have, therefore you are in a
      position to ask better questions. More
      importantly, because agilists deliver working
      software on a regular basis, your stakeholders’
      knowledge of the solution space is much better
      and therefore they can provide more accurate
      information. Hay’s article is interesting, in
      particular his overview of the IT industry is
      useful for understanding why the data community
      is reluctant to adopt modern techniques. Never
      the less, the onus is still on the data community
      to find effective ways to work with evolutionary
      and agile developers – the current status quo clearly isn’t acceptable.

      - Scott

      PS -- My apologies if you've received several copies of this announcement.

      Scott W. Ambler
      Senior Consultant, Ambysoft Inc.
    Your message has been successfully submitted and would be delivered to recipients shortly.