Loading ...
Sorry, an error occurred while loading the content.

Wanna become famous?

Expand Messages
  • Scott Ambler
    At http://www.agiledata.org/essays/renameColumn.html I ve posted an example of how database refactorings are described in mine and Pramod s book Refactoring
    Message 1 of 16 , Dec 2, 2006
    • 0 Attachment
      At http://www.agiledata.org/essays/renameColumn.html
      I've posted an example of how database refactorings
      are described in mine and Pramod's book "Refactoring
      Databases"
      (http://www.ambysoft.com/books/refactoringDatabases.html).
      The example describes how to rename a column and it
      provides code to implement it in an environment where
      Oracle is used on the back end and Hibernate on the
      front end.

      Here's my proposal. I'd love to see different
      implementations published of this refactoring for
      different databases (e.g. DB2, SQLServer, Sybase,
      MySQL, ...) and possibly for different front end
      languages (C#, Java code, ...). The main issue is on
      the database side of things, but having non-Hibernate
      examples would be good too.

      It would be great to have write-ups showing how to use
      the various DB refactoring tools that are now coming
      out. I highly recommend including screen shots.

      If you choose to write up the example and post it on
      the web, I will definitely link to it from my page.
      If you can't host it yourself, I'm sure that we can
      find somewhere to post it, likely at www.agiledata.org
      or www.databaserefactoring.com. We'll worry about
      these logistics when we get to them.

      What's in it for you? The glory of being published!
      Ok, that's not much, but it is sort of cool.

      By showing a variety of implementations of the same
      thing, we can show to the world that agile database
      techniques are applicable to a wide range of
      environments.

      - Scott

      Scott W. Ambler
      Practice Leader Agile Development, IBM Methods Group
      http://www-306.ibm.com/software/rational/bios/ambler.html

      __________________________________________________
      Do You Yahoo!?
      Tired of spam? Yahoo! Mail has the best spam protection around
      http://mail.yahoo.com
    • Adrian Walker
      Hi Scott -- Thanks for your database refactoring example. As I understand it, the example is about ensuring business continuity, while renaming a column in one
      Message 2 of 16 , Dec 3, 2006
      • 0 Attachment
        Hi Scott --

        Thanks for your database refactoring example.

        As I understand it, the example is about ensuring business continuity, while
        renaming a column in one or more tables, using date-activated triggers.

        Apologies if you have covered this elsewhere in your books, but there's an
        architecture that makes column-rename, and perhaps other refactorings, much
        easier.

        The basic idea is that any application touching the database does a quick
        schema lookup before proceeding. The schema tells the application if
        there has been a column name change. (There is a cost, but perhaps this can
        be minimized by caching the schema and checking a change flag.)

        For example, our Internet Business Logic System talks to DBMSs via rules
        like this one.

        url:reengineeringllc.com dbms:mysql dbname:bar tablename:T6
        port:3306 id:m2q password:f1b2

        --------------------------------------------------------------------------------------------------------------------------------------------------
        estimated demand this-id in this-region is for this-qty gallons of
        this-finished-product in this-month of this-yr

        Reference to the columns of the table T6 is by position rather than by
        name. When the rule is used, a schema lookup finds the current column
        names. There's more about this in [1].

        Refactoring should be particularly easy in this case, because the SQL is
        generated by the system, rather than hard coded by a person.

        How does this sound to you?

        Cheers, -- Adrian

        [1]
        www.reengineeringllc.com/Oil_Industry_Supply_Chain_by_Kowalski_and_Walker.pdf

        Internet Business Logic (R)
        Executable open vocabulary English
        Online at www.reengineeringllc.com
        Shared use is free
        Adrian Walker
        Reengineering
        Phone: USA 860 830 2085



        On 12/2/06, Scott Ambler <scottwambler@...> wrote:
        >
        > At http://www.agiledata.org/essays/renameColumn.html
        > I've posted an example of how database refactorings
        > are described in mine and Pramod's book "Refactoring
        > Databases"
        > (http://www.ambysoft.com/books/refactoringDatabases.html).
        > The example describes how to rename a column and it
        > provides code to implement it in an environment where
        > Oracle is used on the back end and Hibernate on the
        > front end.
        >
        > Here's my proposal. I'd love to see different
        > implementations published of this refactoring for
        > different databases (e.g. DB2, SQLServer, Sybase,
        > MySQL, ...) and possibly for different front end
        > languages (C#, Java code, ...). The main issue is on
        > the database side of things, but having non-Hibernate
        > examples would be good too.
        >
        > It would be great to have write-ups showing how to use
        > the various DB refactoring tools that are now coming
        > out. I highly recommend including screen shots.
        >
        > If you choose to write up the example and post it on
        > the web, I will definitely link to it from my page.
        > If you can't host it yourself, I'm sure that we can
        > find somewhere to post it, likely at www.agiledata.org
        > or www.databaserefactoring.com. We'll worry about
        > these logistics when we get to them.
        >
        > What's in it for you? The glory of being published!
        > Ok, that's not much, but it is sort of cool.
        >
        > By showing a variety of implementations of the same
        > thing, we can show to the world that agile database
        > techniques are applicable to a wide range of
        > environments.
        >
        > - Scott
        >
        > Scott W. Ambler
        > Practice Leader Agile Development, IBM Methods Group
        > http://www-306.ibm.com/software/rational/bios/ambler.html
        >
        > __________________________________________________
        > Do You Yahoo!?
        > Tired of spam? Yahoo! Mail has the best spam protection around
        > http://mail.yahoo.com
        >
        >


        [Non-text portions of this message have been removed]
      • Scott Ambler
        Sounds good to me. Would you be willing to write up something overviewing the approach, then providing a specific discussion of renaming a column, then
        Message 3 of 16 , Dec 4, 2006
        • 0 Attachment
          Sounds good to me. Would you be willing to write up
          something overviewing the approach, then providing a
          specific discussion of renaming a column, then perhaps
          a discussion something like "and here are other
          refactorings you could do"?

          - Scott

          --- Adrian Walker <adriandwalker@...> wrote:

          > Hi Scott --
          >
          > Thanks for your database refactoring example.
          >
          > As I understand it, the example is about ensuring
          > business continuity, while
          > renaming a column in one or more tables, using
          > date-activated triggers.
          >
          > Apologies if you have covered this elsewhere in your
          > books, but there's an
          > architecture that makes column-rename, and perhaps
          > other refactorings, much
          > easier.
          >
          > The basic idea is that any application touching the
          > database does a quick
          > schema lookup before proceeding. The schema
          > tells the application if
          > there has been a column name change. (There is a
          > cost, but perhaps this can
          > be minimized by caching the schema and checking a
          > change flag.)
          >
          > For example, our Internet Business Logic System
          > talks to DBMSs via rules
          > like this one.
          >
          > url:reengineeringllc.com dbms:mysql
          > dbname:bar tablename:T6
          > port:3306 id:m2q password:f1b2
          >
          >
          --------------------------------------------------------------------------------------------------------------------------------------------------
          > estimated demand this-id in this-region is for
          > this-qty gallons of
          > this-finished-product in this-month of this-yr
          >
          > Reference to the columns of the table T6 is by
          > position rather than by
          > name. When the rule is used, a schema lookup finds
          > the current column
          > names. There's more about this in [1].
          >
          > Refactoring should be particularly easy in this
          > case, because the SQL is
          > generated by the system, rather than hard coded by a
          > person.
          >
          > How does this sound to you?
          >
          >
          > Cheers, -- Adrian
          >
          > [1]
          >
          www.reengineeringllc.com/Oil_Industry_Supply_Chain_by_Kowalski_and_Walker.pdf
          >
          > Internet Business Logic (R)
          > Executable open vocabulary English
          > Online at www.reengineeringllc.com
          > Shared use is free
          > Adrian Walker
          > Reengineering
          > Phone: USA 860 830 2085
          >
          >
          >
          > On 12/2/06, Scott Ambler <scottwambler@...>
          > wrote:
          > >
          > > At
          > http://www.agiledata.org/essays/renameColumn.html
          > > I've posted an example of how database
          > refactorings
          > > are described in mine and Pramod's book
          > "Refactoring
          > > Databases"
          > >
          >
          (http://www.ambysoft.com/books/refactoringDatabases.html).
          > > The example describes how to rename a column and
          > it
          > > provides code to implement it in an environment
          > where
          > > Oracle is used on the back end and Hibernate on
          > the
          > > front end.
          > >
          > > Here's my proposal. I'd love to see different
          > > implementations published of this refactoring for
          > > different databases (e.g. DB2, SQLServer, Sybase,
          > > MySQL, ...) and possibly for different front end
          > > languages (C#, Java code, ...). The main issue is
          > on
          > > the database side of things, but having
          > non-Hibernate
          > > examples would be good too.
          > >
          > > It would be great to have write-ups showing how to
          > use
          > > the various DB refactoring tools that are now
          > coming
          > > out. I highly recommend including screen shots.
          > >
          > > If you choose to write up the example and post it
          > on
          > > the web, I will definitely link to it from my
          > page.
          > > If you can't host it yourself, I'm sure that we
          > can
          > > find somewhere to post it, likely at
          > www.agiledata.org
          > > or www.databaserefactoring.com. We'll worry about
          > > these logistics when we get to them.
          > >
          > > What's in it for you? The glory of being
          > published!
          > > Ok, that's not much, but it is sort of cool.
          > >
          > > By showing a variety of implementations of the
          > same
          > > thing, we can show to the world that agile
          > database
          > > techniques are applicable to a wide range of
          > > environments.
          > >
          > > - Scott
          > >
          > > Scott W. Ambler
          > > Practice Leader Agile Development, IBM Methods
          > Group
          > >
          >
          http://www-306.ibm.com/software/rational/bios/ambler.html
          > >
          > > __________________________________________________
          > > Do You Yahoo!?
          > > Tired of spam? Yahoo! Mail has the best spam
          > protection around
          > > http://mail.yahoo.com
          > >
          > >
          >
          >
          > [Non-text portions of this message have been
          > removed]
          >
          >
          >
          >
          > Yahoo! Groups Links
          >
          > (Yahoo! ID required)
          >
          > mailto:agileDatabases-fullfeatured@yahoogroups.com
          >
          >
          >


          Scott W. Ambler
          Practice Leader Agile Development, IBM Methods Group
          http://www-306.ibm.com/software/rational/bios/ambler.html

          __________________________________________________
          Do You Yahoo!?
          Tired of spam? Yahoo! Mail has the best spam protection around
          http://mail.yahoo.com
        • complystill
          Hi, When all applications are writen in Java, there s an easier way I think. I ve developed a new Object Oriented Database - Ableverse The Object Base
          Message 4 of 16 , Dec 9, 2006
          • 0 Attachment
            Hi,

            When all applications are writen in Java, there's an easier way I
            think.
            I've developed a new Object Oriented Database - Ableverse The
            Object Base (http://tob.ableverse.org). In fact I'm right now
            finishing its tutorial (http://www.ableverse.org/tutorials/tob/)

            For the renaming problem, if all applications are based on TOB, it
            is almost as easy as refactoring a java instance field. We can define
            the customer persistent class as:

            @...(typeInSwap="Customer")
            public class Customer extends av.tob.TheObject
            {
            @TypeSpec(precision="200")
            private String fname;

            public Customer(String firstName, String lastName)
            {
            this.fname = firstName;
            }

            public String getFirstName()
            {
            return fname;
            }

            @Writing
            public void setFirstName(String firstName)
            {
            this.fname = firstName;
            }
            }

            Renaming the fname field of Customer class will be rather easy with
            Eclipse. Then write a simple tool class:

            import av.tob.*;
            import av.tob.rdb.*;

            public class RefactorCustomerFirstName
            {
            public static void main(String[] args)
            {
            TheObjectBase<SQLQuerier> tob =
            TheObjectBase.create(SQLQuerier.class);

            tob.querier.update("UPDATE CUSTOMER SET "
            + CustomerQ.FIRSTNAME
            + " = FNAME");
            tob.shudown();
            }
            }

            Rebuild the persistent model classes (recompiling with APT will
            generate updated supporting classes), together with the above tool
            class.

            Then run this tool class:
            1. Starting the TOB instance with
            tob = TheObjectBase.create(SQLQuerier.class);
            will cause a sql script with 'ADD COLUMN' generated and
            automatically executed.
            2. Then executing
            tob.querier.update("UPDATE CUSTOMER SET "
            + CustomerQ.FIRSTNAME
            + " = FNAME");
            will copy the column value.

            Finally distribute the rebuilt persistent model jar to other
            applications if necessary.

            And all done.

            For concern about foreign key references, TOB handles 'JOINs' some
            different, and the equivalent might be @Tie fields defined at
            relation classes (extending av.tob.TheRelation). If a tie field name
            is changed, other code referencing it will not compile and apt
            compiler will complain that the old tie field can not be found, then
            resolving this error will get those code aligned with new persistent
            model.

            But this assumes all applications are developed and run upon Java
            5.0, and since TOB has not been widely used (even not widely
            announced yet), I'd like to publish these refactoring information a
            little later.

            If someone is interested to run a TOB based code piece, download
            http://www.ableverse.org/patterns/av5patterns.zip But this package is
            not publicly published yet, sorry for the lack of detailed steps for
            building and running. Basically it is an Eclipse project, you can
            directly import it into eclipse, then launch the build.xml within
            eclipse to compile it, but note it will require apache ant 1.7.0,
            should be first downloaded and unpacked separately. If you know well
            about java dev with eclipse, it's
            no hard to get through. Please check and modify meta/tob.meta before
            run each runnable classes, it needs proper configuration to a
            relational database connection, like Oracle, MySQL, hsqldb, h2
            If you can't find TOB downloads, here it is:
            http://www.ableverse.com/download-free.jsp

            Interests on TOB's production quality for now should be redirected to
            evaluation of the WoW Project (http://wow.dev.java.net), it has a
            live demo at: http://www.webofweb.net

            Though all TOBs of current release run upon relational databases, it
            is better considered as a database on its own foot, for reasons
            please keep an eye on http://tob.ableverse.org and http://
            www.ableverse.org/tutorials/tob/ which is work in progress. TOB was
            designed with agile development in mind, i.e. automatic schema
            alignment without container shutdown, and hope it can meet
            requirements of people on this list, and help some in future agile
            evolvement.

            Thanks to read down here!

            With Regards,
            - Compl (http://www.ableverse.org Programming By Nature)


            --- In agileDatabases@yahoogroups.com, Scott Ambler
            <scottwambler@...> wrote:
            >
            > At http://www.agiledata.org/essays/renameColumn.html
            > I've posted an example of how database refactorings
            > are described in mine and Pramod's book "Refactoring
            > Databases"
            > (http://www.ambysoft.com/books/refactoringDatabases.html).
            > The example describes how to rename a column and it
            > provides code to implement it in an environment where
            > Oracle is used on the back end and Hibernate on the
            > front end.
            >
            > Here's my proposal. I'd love to see different
            > implementations published of this refactoring for
            > different databases (e.g. DB2, SQLServer, Sybase,
            > MySQL, ...) and possibly for different front end
            > languages (C#, Java code, ...). The main issue is on
            > the database side of things, but having non-Hibernate
            > examples would be good too.
            >
            > It would be great to have write-ups showing how to use
            > the various DB refactoring tools that are now coming
            > out. I highly recommend including screen shots.
            >
            > If you choose to write up the example and post it on
            > the web, I will definitely link to it from my page.
            > If you can't host it yourself, I'm sure that we can
            > find somewhere to post it, likely at www.agiledata.org
            > or www.databaserefactoring.com. We'll worry about
            > these logistics when we get to them.
            >
            > What's in it for you? The glory of being published!
            > Ok, that's not much, but it is sort of cool.
            >
            > By showing a variety of implementations of the same
            > thing, we can show to the world that agile database
            > techniques are applicable to a wide range of
            > environments.
            >
            > - Scott
            >
            > Scott W. Ambler
            > Practice Leader Agile Development, IBM Methods Group
            > http://www-306.ibm.com/software/rational/bios/ambler.html
            >
            > __________________________________________________
            > Do You Yahoo!?
            > Tired of spam? Yahoo! Mail has the best spam protection around
            > http://mail.yahoo.com
            >
          • Scott Ambler
            Sounds interesting. When you ve decided to publish your solution let me know so I can link to it. - Scott ...
            Message 5 of 16 , Dec 11, 2006
            • 0 Attachment
              Sounds interesting. When you've decided to publish
              your solution let me know so I can link to it.

              - Scott
              --- complystill <complystill@...> wrote:

              > Hi,
              >
              > When all applications are writen in Java, there's
              > an easier way I
              > think.
              > I've developed a new Object Oriented Database -
              > Ableverse The
              > Object Base (http://tob.ableverse.org). In fact I'm
              > right now
              > finishing its tutorial
              > (http://www.ableverse.org/tutorials/tob/)
              >
              > For the renaming problem, if all applications are
              > based on TOB, it
              > is almost as easy as refactoring a java instance
              > field. We can define
              > the customer persistent class as:
              >
              > @...(typeInSwap="Customer")
              > public class Customer extends av.tob.TheObject
              > {
              > @TypeSpec(precision="200")
              > private String fname;
              >
              > public Customer(String firstName, String
              > lastName)
              > {
              > this.fname = firstName;
              > }
              >
              > public String getFirstName()
              > {
              > return fname;
              > }
              >
              > @Writing
              > public void setFirstName(String firstName)
              > {
              > this.fname = firstName;
              > }
              > }
              >
              > Renaming the fname field of Customer class will be
              > rather easy with
              > Eclipse. Then write a simple tool class:
              >
              > import av.tob.*;
              > import av.tob.rdb.*;
              >
              > public class RefactorCustomerFirstName
              > {
              > public static void main(String[] args)
              > {
              > TheObjectBase<SQLQuerier> tob =
              > TheObjectBase.create(SQLQuerier.class);
              >
              > tob.querier.update("UPDATE CUSTOMER SET "
              > + CustomerQ.FIRSTNAME
              > + " = FNAME");
              > tob.shudown();
              > }
              > }
              >
              > Rebuild the persistent model classes (recompiling
              > with APT will
              > generate updated supporting classes), together with
              > the above tool
              > class.
              >
              > Then run this tool class:
              > 1. Starting the TOB instance with
              > tob =
              > TheObjectBase.create(SQLQuerier.class);
              > will cause a sql script with 'ADD COLUMN'
              > generated and
              > automatically executed.
              > 2. Then executing
              > tob.querier.update("UPDATE CUSTOMER SET "
              > + CustomerQ.FIRSTNAME
              > + " = FNAME");
              > will copy the column value.
              >
              > Finally distribute the rebuilt persistent model jar
              > to other
              > applications if necessary.
              >
              > And all done.
              >
              > For concern about foreign key references, TOB
              > handles 'JOINs' some
              > different, and the equivalent might be @Tie fields
              > defined at
              > relation classes (extending av.tob.TheRelation). If
              > a tie field name
              > is changed, other code referencing it will not
              > compile and apt
              > compiler will complain that the old tie field can
              > not be found, then
              > resolving this error will get those code aligned
              > with new persistent
              > model.
              >
              > But this assumes all applications are developed and
              > run upon Java
              > 5.0, and since TOB has not been widely used (even
              > not widely
              > announced yet), I'd like to publish these
              > refactoring information a
              > little later.
              >
              > If someone is interested to run a TOB based code
              > piece, download
              > http://www.ableverse.org/patterns/av5patterns.zip
              > But this package is
              > not publicly published yet, sorry for the lack of
              > detailed steps for
              > building and running. Basically it is an Eclipse
              > project, you can
              > directly import it into eclipse, then launch the
              > build.xml within
              > eclipse to compile it, but note it will require
              > apache ant 1.7.0,
              > should be first downloaded and unpacked separately.
              > If you know well
              > about java dev with eclipse, it's
              > no hard to get through. Please check and modify
              > meta/tob.meta before
              > run each runnable classes, it needs proper
              > configuration to a
              > relational database connection, like Oracle, MySQL,
              > hsqldb, h2
              > If you can't find TOB downloads, here it is:
              > http://www.ableverse.com/download-free.jsp
              >
              > Interests on TOB's production quality for now should
              > be redirected to
              > evaluation of the WoW Project
              > (http://wow.dev.java.net), it has a
              > live demo at: http://www.webofweb.net
              >
              > Though all TOBs of current release run upon
              > relational databases, it
              > is better considered as a database on its own foot,
              > for reasons
              > please keep an eye on http://tob.ableverse.org and
              > http://
              > www.ableverse.org/tutorials/tob/ which is work in
              > progress. TOB was
              > designed with agile development in mind, i.e.
              > automatic schema
              > alignment without container shutdown, and hope it
              > can meet
              > requirements of people on this list, and help some
              > in future agile
              > evolvement.
              >
              > Thanks to read down here!
              >
              > With Regards,
              > - Compl (http://www.ableverse.org Programming By
              > Nature)
              >
              >
              > --- In agileDatabases@yahoogroups.com, Scott Ambler
              > <scottwambler@...> wrote:
              > >
              > > At
              > http://www.agiledata.org/essays/renameColumn.html
              > > I've posted an example of how database
              > refactorings
              > > are described in mine and Pramod's book
              > "Refactoring
              > > Databases"
              > >
              >
              (http://www.ambysoft.com/books/refactoringDatabases.html).
              > > The example describes how to rename a column and
              > it
              > > provides code to implement it in an environment
              > where
              > > Oracle is used on the back end and Hibernate on
              > the
              > > front end.
              > >
              > > Here's my proposal. I'd love to see different
              > > implementations published of this refactoring for
              > > different databases (e.g. DB2, SQLServer, Sybase,
              > > MySQL, ...) and possibly for different front end
              > > languages (C#, Java code, ...). The main issue is
              > on
              > > the database side of things, but having
              > non-Hibernate
              > > examples would be good too.
              > >
              > > It would be great to have write-ups showing how to
              > use
              > > the various DB refactoring tools that are now
              > coming
              > > out. I highly recommend including screen shots.
              > >
              > > If you choose to write up the example and post it
              > on
              >
              === message truncated ===


              Scott W. Ambler
              Practice Leader Agile Development, IBM Methods Group
              http://www-306.ibm.com/software/rational/bios/ambler.html

              __________________________________________________
              Do You Yahoo!?
              Tired of spam? Yahoo! Mail has the best spam protection around
              http://mail.yahoo.com
            • Light, Kevin
              Scott, I have a few questions for you. Perhaps I am mis-reading what you are trying to say however, based on the following note, I m wondering if the
              Message 6 of 16 , Dec 14, 2006
              • 0 Attachment
                Scott, I have a few questions for you. Perhaps I am mis-reading what
                you are trying to say however, based on the following note, I'm
                wondering if the take-away is that the refactoring techniques you have
                published on your website and in your book haven't actually been
                implemented by you or your co-author. Is that true? If not, then could
                you please provide some details for each implementation. In particular,
                the following information would be of great interest:

                * the size of the database, that is, the number of tables to the
                nearest hundred
                * the number of on-line users, to the nearest hundred
                * the number of rows for each table affected by the re-factoring,
                to the nearest million
                * the amount of time the data base was off-line to effect the
                change(s)
                * the number of views and application programs that required
                change before the database was fully functional again.

                It would also be useful if you could indicate whether any of the
                re-factorings affected fundamental data definitions, data precision or
                granularity. For those components that were affected the method used to
                gain consensus on the change prior to accomplishing it would also be of
                interest.



                cheers

                Kevin Light
                EDS Solutions Development, Data & Applications Integration
                10888 White Rock Road
                mail: 3141 Data Drive
                Rancho Cordova, CA 95670

                + mailto:kevin.light@...


                ________________________________

                From: agileDatabases@yahoogroups.com
                [mailto:agileDatabases@yahoogroups.com] On Behalf Of complystill
                Sent: Saturday, December 09, 2006 2:24 AM
                To: agileDatabases@yahoogroups.com
                Subject: [agileDatabases] Re: Wanna become famous?



                [kcl says:] snip

                --- In agileDatabases@yahoogroups.com
                <mailto:agileDatabases%40yahoogroups.com> , Scott Ambler
                <scottwambler@...> wrote:
                >
                > At http://www.agiledata.org/essays/renameColumn.html
                <http://www.agiledata.org/essays/renameColumn.html>
                > I've posted an example of how database refactorings
                > are described in mine and Pramod's book "Refactoring
                > Databases"
                > (http://www.ambysoft.com/books/refactoringDatabases.html
                <http://www.ambysoft.com/books/refactoringDatabases.html> ).
                > The example describes how to rename a column and it
                > provides code to implement it in an environment where
                > Oracle is used on the back end and Hibernate on the
                > front end.
                >
                > Here's my proposal. I'd love to see different
                > implementations published of this refactoring for
                > different databases (e.g. DB2, SQLServer, Sybase,
                > MySQL, ...) and possibly for different front end
                > languages (C#, Java code, ...). The main issue is on
                > the database side of things, but having non-Hibernate
                > examples would be good too.
                >
                > It would be great to have write-ups showing how to use
                > the various DB refactoring tools that are now coming
                > out. I highly recommend including screen shots.
                >
                > If you choose to write up the example and post it on
                > the web, I will definitely link to it from my page.
                > If you can't host it yourself, I'm sure that we can
                > find somewhere to post it, likely at www.agiledata.org
                > or www.databaserefactoring.com. We'll worry about
                > these logistics when we get to them.
                >
                > What's in it for you? The glory of being published!
                > Ok, that's not much, but it is sort of cool.
                >
                > By showing a variety of implementations of the same
                > thing, we can show to the world that agile database
                > techniques are applicable to a wide range of
                > environments.
                >
                > - Scott
                >
                > Scott W. Ambler
                > Practice Leader Agile Development, IBM Methods Group
                > http://www-306.ibm.com/software/rational/bios/ambler.html
                <http://www-306.ibm.com/software/rational/bios/ambler.html>
                >
                > __________________________________________________
                > Do You Yahoo!?
                > Tired of spam? Yahoo! Mail has the best spam protection around

                > http://mail.yahoo.com <http://mail.yahoo.com>
                >







                [Non-text portions of this message have been removed]
              • Scott Ambler
                ... Have you read the book? It s full of code examples, in fact every single refactoring, and the handfull of non-refactoring transformations that we include,
                Message 7 of 16 , Dec 15, 2006
                • 0 Attachment
                  --- "Light, Kevin" <kevin.light@...> wrote:

                  >
                  >
                  > Scott, I have a few questions for you. Perhaps I am
                  > mis-reading what
                  > you are trying to say however, based on the
                  > following note, I'm
                  > wondering if the take-away is that the refactoring
                  > techniques you have
                  > published on your website and in your book haven't
                  > actually been
                  > implemented by you or your co-author.

                  Have you read the book? It's full of code examples,
                  in fact every single refactoring, and the handfull of
                  non-refactoring transformations that we include, is
                  described with code examples. All of the code
                  examples, however, are in Oracle on the database side
                  of things and either Java or Hibernate on the
                  application side of things (probably a 80/20 split
                  between Java/Hibernate).

                  What I was asking for is if other people would like to
                  show the implementation details for other technical
                  platforms.



                  > Is that true?

                  Not at all.

                  > If not, then could
                  > you please provide some details for each
                  > implementation.
                  > In particular,
                  > the following information would be of great
                  > interest:

                  All of these issues depends on the situation. Both
                  Pramod and I have worked for various clients over the
                  years. The figures below are for myself.

                  >
                  > * the size of the database, that is, the number of
                  > tables to the
                  > nearest hundred

                  Tens of tables to thousands.

                  > * the number of on-line users, to the nearest
                  > hundred

                  Number of users from tens to tens of thousands.


                  > * the number of rows for each table affected by the
                  > re-factoring,
                  > to the nearest million

                  Into the millions (yes, sometimes that's slow). A lot
                  of people like to use the excuse "I couldn't possible
                  fix this table because it's so big". I have to think
                  that because it's so big that the problem is even
                  worse, and therefore you might be even more motivated
                  to fix it. Then again, I don't look for excuses to
                  shirk (sp?) quality.



                  > * the amount of time the data base was off-line to
                  > effect the
                  > change(s)

                  Depends on the SLAs. Sometimes to roll the changes
                  into production we're only allowed to have the DB down
                  for a few minutes, so it makes refactoring larger dbs
                  very difficult.

                  > * the number of views and application programs that
                  > required
                  > change before the database was fully functional
                  > again.

                  Depends.


                  >
                  > It would also be useful if you could indicate
                  > whether any of the
                  > re-factorings affected fundamental data definitions,
                  > data precision or
                  > granularity.

                  Clearly you haven't read the book. The data quality
                  refactorings are pretty much guaranteed to do that.

                  > For those components that were
                  > affected the method used to
                  > gain consensus on the change prior to accomplishing
                  > it would also be of
                  > interest.

                  Read Chapters 2 - 5 for various strategies.

                  - Scott

                  Scott W. Ambler
                  Practice Leader Agile Development, IBM Methods Group
                  http://www-306.ibm.com/software/rational/bios/ambler.html

                  __________________________________________________
                  Do You Yahoo!?
                  Tired of spam? Yahoo! Mail has the best spam protection around
                  http://mail.yahoo.com
                • Pramod Sadalage
                  Kevin, The refactoring techniques we have written about where not just dreamed up, we had been doing them for years over multiple projects at many different
                  Message 8 of 16 , Dec 15, 2006
                  • 0 Attachment
                    Kevin,

                    The refactoring techniques we have written about where not just dreamed up,
                    we had been doing them for years over multiple projects at many different
                    clients using many different database environments and many different
                    application languages. The book is a compilation of patterns we found where
                    common among the work that we performed over the last 5-7 years.

                    What Scott and I are looking at is how people implemented a given
                    refactoring so that there is more reference material for the community to
                    use and benefit from.

                    Thanks
                    Pramod
                    Co-Author "Refactoring Databases: Evolutionary Database Design"
                    www.databaserefactoring.com


                    On 12/14/06, Light, Kevin <kevin.light@...> wrote:
                    >
                    >
                    >
                    > Scott, I have a few questions for you. Perhaps I am mis-reading what
                    > you are trying to say however, based on the following note, I'm
                    > wondering if the take-away is that the refactoring techniques you have
                    > published on your website and in your book haven't actually been
                    > implemented by you or your co-author. Is that true? If not, then could
                    > you please provide some details for each implementation. In particular,
                    > the following information would be of great interest:
                    >
                    > * the size of the database, that is, the number of tables to the
                    > nearest hundred
                    > * the number of on-line users, to the nearest hundred
                    > * the number of rows for each table affected by the re-factoring,
                    > to the nearest million
                    > * the amount of time the data base was off-line to effect the
                    > change(s)
                    > * the number of views and application programs that required
                    > change before the database was fully functional again.
                    >
                    > It would also be useful if you could indicate whether any of the
                    > re-factorings affected fundamental data definitions, data precision or
                    > granularity. For those components that were affected the method used to
                    > gain consensus on the change prior to accomplishing it would also be of
                    > interest.
                    >
                    > cheers
                    >
                    > Kevin Light
                    > EDS Solutions Development, Data & Applications Integration
                    > 10888 White Rock Road
                    > mail: 3141 Data Drive
                    > Rancho Cordova, CA 95670
                    >
                    > + mailto:kevin.light@... <kevin.light%40eds.com>
                    >
                    > ________________________________
                    >
                    > From: agileDatabases@yahoogroups.com <agileDatabases%40yahoogroups.com>
                    > [mailto:agileDatabases@yahoogroups.com <agileDatabases%40yahoogroups.com>]
                    > On Behalf Of complystill
                    > Sent: Saturday, December 09, 2006 2:24 AM
                    > To: agileDatabases@yahoogroups.com <agileDatabases%40yahoogroups.com>
                    > Subject: [agileDatabases] Re: Wanna become famous?
                    >
                    >
                    >
                    > [kcl says:] snip
                    >
                    > --- In agileDatabases@yahoogroups.com <agileDatabases%40yahoogroups.com>
                    > <mailto:agileDatabases%40yahoogroups.com> , Scott Ambler
                    > <scottwambler@...> wrote:
                    > >
                    > > At http://www.agiledata.org/essays/renameColumn.html
                    > <http://www.agiledata.org/essays/renameColumn.html>
                    > > I've posted an example of how database refactorings
                    > > are described in mine and Pramod's book "Refactoring
                    > > Databases"
                    > > (http://www.ambysoft.com/books/refactoringDatabases.html
                    > <http://www.ambysoft.com/books/refactoringDatabases.html> ).
                    > > The example describes how to rename a column and it
                    > > provides code to implement it in an environment where
                    > > Oracle is used on the back end and Hibernate on the
                    > > front end.
                    > >
                    > > Here's my proposal. I'd love to see different
                    > > implementations published of this refactoring for
                    > > different databases (e.g. DB2, SQLServer, Sybase,
                    > > MySQL, ...) and possibly for different front end
                    > > languages (C#, Java code, ...). The main issue is on
                    > > the database side of things, but having non-Hibernate
                    > > examples would be good too.
                    > >
                    > > It would be great to have write-ups showing how to use
                    > > the various DB refactoring tools that are now coming
                    > > out. I highly recommend including screen shots.
                    > >
                    > > If you choose to write up the example and post it on
                    > > the web, I will definitely link to it from my page.
                    > > If you can't host it yourself, I'm sure that we can
                    > > find somewhere to post it, likely at www.agiledata.org
                    > > or www.databaserefactoring.com. We'll worry about
                    > > these logistics when we get to them.
                    > >
                    > > What's in it for you? The glory of being published!
                    > > Ok, that's not much, but it is sort of cool.
                    > >
                    > > By showing a variety of implementations of the same
                    > > thing, we can show to the world that agile database
                    > > techniques are applicable to a wide range of
                    > > environments.
                    > >
                    > > - Scott
                    > >
                    > > Scott W. Ambler
                    > > Practice Leader Agile Development, IBM Methods Group
                    > > http://www-306.ibm.com/software/rational/bios/ambler.html
                    > <http://www-306.ibm.com/software/rational/bios/ambler.html>
                    > >
                    > > __________________________________________________
                    > > Do You Yahoo!?
                    > > Tired of spam? Yahoo! Mail has the best spam protection around
                    >
                    > > http://mail.yahoo.com <http://mail.yahoo.com>
                    > >
                    >
                    >
                    >
                    >
                    >
                    > [Non-text portions of this message have been removed]
                    >
                    >
                    >


                    [Non-text portions of this message have been removed]
                  • bob_corrick
                    ... dreamed up, we had been doing them for years over multiple projects at many different clients using many different database environments and many different
                    Message 9 of 16 , Dec 15, 2006
                    • 0 Attachment
                      --- "Pramod Sadalage" <pom.biker@...> wrote: <extract>
                      > The refactoring techniques we have written about where not just
                      dreamed up, we had been doing them for years over multiple projects
                      at many different clients using many different database environments
                      and many different application languages. </extract>

                      > On 12/14/06, Light, Kevin <kevin.light@...> wrote: <extract>
                      > > the following information would be of great interest:
                      > >
                      > > * the size of the database...
                      > > * the number of on-line users...
                      > > * the number of rows for each table affected...
                      > > * the amount of time the data base was off-line...
                      > > * the number of views and application programs... </extract>

                      As the example that started this thread says, "The primary trade-off
                      is the cost of refactoring the external applications that access the
                      column versus the improved readability and/or consistency provided
                      by the new name."

                      I too would be interested in some "war stories" to back up the
                      patterns.

                      regards
                      Bob Corrick
                    • Light, Kevin
                      Uh, Scott, that is not what I asked. You have insisted on dm-discuss that these are proven techniques, used by database practitioners on a regular basis. I
                      Message 10 of 16 , Dec 15, 2006
                      • 0 Attachment
                        Uh, Scott, that is not what I asked. You have insisted on dm-discuss
                        that these are proven techniques, used by database practitioners on a
                        regular basis. I am simply asking you to provide some concrete
                        examples. All I have ever seen from you on this (and other) subject(s)
                        are links to your website and recommendations that I read your book(s).
                        Can you provide even one example to answer the questions I have posed?

                        I have been involved in data warehouse re-engineering projects and I can
                        tell you that they were not simple, not fast and not cheap. I would
                        dearly love to know how to exercise these engagements faster, better and
                        cheaper. That means real-life case studies and detailed examples of the
                        strategy, process and techniques that have proven to be successful. Are
                        there detailed case studies in your book for each of the techniques that
                        you propose? If yes, I would hope that they include the information
                        that I have requested and that you would be willing to share at least
                        one of the more challenging examples.

                        Thanks in advance.




                        cheers

                        Kevin Light
                        EDS Solutions Development, Data & Applications Integration
                        10888 White Rock Road
                        mail: 3141 Data Drive
                        Rancho Cordova, CA 95670

                        + mailto:kevin.light@...


                        ________________________________

                        From: agileDatabases@yahoogroups.com
                        [mailto:agileDatabases@yahoogroups.com] On Behalf Of Scott Ambler
                        Sent: Friday, December 15, 2006 4:21 AM
                        To: agileDatabases@yahoogroups.com
                        Subject: RE: [agileDatabases] Re: Wanna become famous?




                        --- "Light, Kevin" <kevin.light@...
                        <mailto:kevin.light%40eds.com> > wrote:

                        >
                        >
                        > Scott, I have a few questions for you. Perhaps I am
                        > mis-reading what
                        > you are trying to say however, based on the
                        > following note, I'm
                        > wondering if the take-away is that the refactoring
                        > techniques you have
                        > published on your website and in your book haven't
                        > actually been
                        > implemented by you or your co-author.

                        Have you read the book? It's full of code examples,
                        in fact every single refactoring, and the handfull of
                        non-refactoring transformations that we include, is
                        described with code examples. All of the code
                        examples, however, are in Oracle on the database side
                        of things and either Java or Hibernate on the
                        application side of things (probably a 80/20 split
                        between Java/Hibernate).

                        What I was asking for is if other people would like to
                        show the implementation details for other technical
                        platforms.

                        > Is that true?

                        Not at all.

                        > If not, then could
                        > you please provide some details for each
                        > implementation.
                        > In particular,
                        > the following information would be of great
                        > interest:

                        All of these issues depends on the situation. Both
                        Pramod and I have worked for various clients over the
                        years. The figures below are for myself.

                        >
                        > * the size of the database, that is, the number of
                        > tables to the
                        > nearest hundred

                        Tens of tables to thousands.

                        > * the number of on-line users, to the nearest
                        > hundred

                        Number of users from tens to tens of thousands.

                        > * the number of rows for each table affected by the
                        > re-factoring,
                        > to the nearest million

                        Into the millions (yes, sometimes that's slow). A lot
                        of people like to use the excuse "I couldn't possible
                        fix this table because it's so big". I have to think
                        that because it's so big that the problem is even
                        worse, and therefore you might be even more motivated
                        to fix it. Then again, I don't look for excuses to
                        shirk (sp?) quality.

                        > * the amount of time the data base was off-line to
                        > effect the
                        > change(s)

                        Depends on the SLAs. Sometimes to roll the changes
                        into production we're only allowed to have the DB down
                        for a few minutes, so it makes refactoring larger dbs
                        very difficult.

                        > * the number of views and application programs that
                        > required
                        > change before the database was fully functional
                        > again.

                        Depends.

                        >
                        > It would also be useful if you could indicate
                        > whether any of the
                        > re-factorings affected fundamental data definitions,
                        > data precision or
                        > granularity.

                        Clearly you haven't read the book. The data quality
                        refactorings are pretty much guaranteed to do that.

                        > For those components that were
                        > affected the method used to
                        > gain consensus on the change prior to accomplishing
                        > it would also be of
                        > interest.

                        Read Chapters 2 - 5 for various strategies.

                        - Scott

                        Scott W. Ambler
                        Practice Leader Agile Development, IBM Methods Group
                        http://www-306.ibm.com/software/rational/bios/ambler.html
                        <http://www-306.ibm.com/software/rational/bios/ambler.html>

                        __________________________________________________
                        Do You Yahoo!?
                        Tired of spam? Yahoo! Mail has the best spam protection around
                        http://mail.yahoo.com <http://mail.yahoo.com>






                        [Non-text portions of this message have been removed]
                      • Scott Ambler
                        That s pretty much the topic of Chapter 5 in the book. We present a series of strategies which we picked up over the years at various clients. Chapter 3
                        Message 11 of 16 , Dec 15, 2006
                        • 0 Attachment
                          That's pretty much the topic of Chapter 5 in the book.
                          We present a series of strategies which we picked up
                          over the years at various clients.

                          Chapter 3 describes in detail how to refactor
                          databases from the point of view of a highly coupled
                          (e.g. many, many systems are accessing the db) db
                          where the external systems are on heterogeneous
                          platforms.

                          Chapter 4 describes how to deploy refactorings into
                          production.

                          You're absolutely right, the nasty issue is organizing
                          the development teams of the external systems to
                          refactor their own code. We cover that issue, and
                          others, in Chapter 5. The focus, however, is on the
                          database aspects and ensuring that the right things
                          happen there. There are other very good books written
                          about code refactoring and legacy application
                          refactoring which we refer people to.

                          On the database side, refactorings are clearly
                          complicated by volume and transaction rate, but then
                          again, what database-oriented activity isn't
                          complicated by those issues. We show in the book how
                          to address the coupling issues with the external
                          systems (and the internal coupling issues for that
                          matter).

                          - Scott
                          --- bob_corrick <bob.corrick@...> wrote:

                          <snip>
                          > As the example that started this thread says, "The
                          > primary trade-off
                          > is the cost of refactoring the external applications
                          > that access the
                          > column versus the improved readability and/or
                          > consistency provided
                          > by the new name."
                          <snip>

                          Scott W. Ambler
                          Practice Leader Agile Development, IBM Methods Group
                          http://www-306.ibm.com/software/rational/bios/ambler.html

                          __________________________________________________
                          Do You Yahoo!?
                          Tired of spam? Yahoo! Mail has the best spam protection around
                          http://mail.yahoo.com
                        • Scott Ambler
                          ... Yes, many of the people on this list perform agile database techniques on a regular basis. If you re polite, perhaps they ll tell you about them. ... See
                          Message 12 of 16 , Dec 15, 2006
                          • 0 Attachment
                            --- "Light, Kevin" <kevin.light@...> wrote:

                            > Uh, Scott, that is not what I asked. You have
                            > insisted on dm-discuss
                            > that these are proven techniques, used by database
                            > practitioners on a
                            > regular basis.

                            Yes, many of the people on this list perform agile
                            database techniques on a regular basis. If you're
                            polite, perhaps they'll tell you about them.


                            > I am simply asking you to provide
                            > some concrete
                            > examples.

                            See the case study below.

                            > All I have ever seen from you on this
                            > (and other) subject(s)
                            > are links to your website and recommendations that I
                            > read your book(s).

                            Yes, I have invested significant time to share my
                            experiences on the web. When I have already written
                            something up, I quite frankly don't see the need to do
                            so again. If you're not willing to click on the links
                            then that's your decision, but please don't whine to
                            me about it.


                            > Can you provide even one example to answer the
                            > questions I have posed?

                            See below.

                            >
                            > I have been involved in data warehouse
                            > re-engineering projects and I can
                            > tell you that they were not simple, not fast and not
                            > cheap.

                            Perhaps you should have applied some of the techniques
                            that I describe at www.agiledata.org ?

                            > I would
                            > dearly love to know how to exercise these
                            > engagements faster, better and
                            > cheaper.

                            That means you're going to have to do some of the
                            reading that I've suggested, sorry about that.

                            > That means real-life case studies and
                            > detailed examples of the
                            > strategy, process and techniques that have proven to
                            > be successful.

                            Detailed discussion of strategies, process and
                            techniques can be found at www.agiledata.org.

                            Case studies can be found from
                            http://www.agiledata.org/essays/caseStudies.html
                            (BTW, if anyone knows of any that I've missed, please
                            let me know as I'm happy to add new links).

                            Sorry Kevin, but I'm really not convinced that the
                            problem is that I haven't provided the material that
                            you're asking for.

                            > Are
                            > there detailed case studies in your book for each of
                            > the techniques that
                            > you propose? If yes, I would hope that they include
                            > the information
                            > that I have requested and that you would be willing
                            > to share at least
                            > one of the more challenging examples.

                            The book shares many experiences, but it's not a case
                            study book. It's a technical reference.

                            Here's a case study for you:

                            Client:
                            Large retailer that needed to extend an existing
                            web-based app
                            5-600 people in IT

                            Technical environment:
                            Oracle 8i database
                            30+ known applications accessing it, written in Java,
                            VB, COBOL, C/C++. At the beginning of the project
                            there was an estimated 10-15 other systems accessing
                            it that were "rumored" to exist. Throughout the
                            project, at least when I was there, we flushed out 8
                            of them.

                            Interesting Database Stats:
                            Around 350 tables, although about 200 were relatively
                            static reference tables
                            Averaged about 500,000 transactions a day, although
                            during holiday seasons that peaked at around 3 million
                            a day
                            Live data was kept for 6 weeks until it was archived,
                            where it was kept ad infinitum from what I could tell

                            What we did technically:
                            - Updated an existing app in time for the coming
                            holiday season. Had to be online by Oct 15 that year
                            otherwise we had missed the window.
                            - Needed to add new functionality to compete with
                            other retailers who quite frankly were farther along
                            in the e-commerce game
                            - Existing database had a few "quality challenges"
                            that needed to be addressed
                            - Internal staff had failed the previous year and they
                            had to back out. For some reason the business folks
                            were upset about that.
                            - We updated the app, putting a testing framework
                            (JUnit) and test suite in place. We were firm
                            believers in TDD, see
                            http://www.agiledata.org/essays/tdd.html
                            - Because we were taking a TDD-based approach to app
                            dev, we decided to do the same with database dev. So,
                            we did database refactoring
                            (http://www.agiledata.org/essays/databaseRefactoring.html)
                            and regression testing
                            (http://www.agiledata.org/essays/databaseTesting.html)
                            as a result. Clicking on the URLs leads to detailed
                            process descriptions.
                            - We delivered our app, barely, on time with the new
                            functionality. We also did a significant clean up of
                            the app itself and of the database. We could show
                            increasing velocity by the end of the project, as well
                            as significantly decreasing defect reports (on the app
                            at least).
                            - As a result of the database testing we identified a
                            very large number of data quality challenges. The
                            company pretty much knew that they had problems, but
                            didn't know the extent of them.
                            - We did about 50 database refactorings, many of which
                            were structural. We fixed some serious data quality
                            problems, the ones that affected what we were doing,
                            but not all of them (we did report them though via
                            their defect reporting system).
                            - We passed final system testing with flying colors.
                            The QA folks said it was one of the most successful
                            implementations that they'd ever seen. Apparently,
                            continuous regression testing and refactoring result
                            in significantly higher quality, who would have
                            figured? ;-)
                            - The business folks were very happy with us.

                            What we did organizationally:
                            - We had one of their DBAs on the project initially.
                            This person wanted to work in a traditional manner.
                            Although we tried explaining the techniques to him he
                            refused to work that way and insisted on working on
                            his own. The PM, a client employee, cut him from the
                            team after two weeks of that nonsense.
                            - The team started doing the data stuff themselves.
                            The data group refused to provide guidelines (naming
                            conventions, ...), tool access, or even database
                            access, insisting that they were the only ones
                            qualified to do that work.
                            - We decided to reverse engineer the naming
                            conventions from the existing data models which
                            appeared on the walls throughout the office. I had a
                            legal copy of ERWin and TOAD on my laptop so tools
                            weren't an issue. The PM had a "strategy meeting"
                            with the head of the data group, the CIO, and the CEO.
                            We had full DB access that afternoon. It took a
                            couple of more days to set up the sandboxes we needed,
                            but that was more of a lack of hardware issue anyway.
                            - By that time the data group wanted to get back in on
                            the action after the C-level folks indicated that we
                            had their full support. Politics ensued, but in the
                            end the data group couldn't explain to us how they
                            could actually help our team. Another "strategy
                            meeting" occurred. We heard little from the data
                            group after that.
                            - When we went to deploy the data group got involved
                            again, claiming that the system could go into
                            production because we didn't follow their processes
                            and therefore couldn't have delivered. I pointed out
                            that the previous year their processes had in fact
                            been followed and the team hadn't delivered. They
                            didn't appreciate that observation for some reason.
                            Another strategy meeting was held were it was
                            determined that the data group had one week to review
                            our work and report back to the steering committee
                            (which had been set up part way through the project as
                            the result of politicking by the data group).
                            - The results came back, revealing that two minor bugs
                            occurred in the updated version of the schema (we'd
                            misnamed two columns). The data group tried billing
                            the project for the review work (internal funny
                            money). I pointed out that this wasn't fair because
                            the data group had insisted on the review, it didn't
                            find anything of substance, and what it did find was
                            their fault anyway because they hadn't provided the
                            naming guidelines which we'd requested several times.
                            The data group wasn't allowed to bill against our
                            project, and they didn't appreciate that for some
                            reason. We also fixed both bugs (renaming columns is
                            pretty straightforward, see
                            http://www.agiledata.org/essays/renameColumn.html ).

                            - Scott




                            Scott W. Ambler
                            Practice Leader Agile Development, IBM Methods Group
                            http://www-306.ibm.com/software/rational/bios/ambler.html

                            __________________________________________________
                            Do You Yahoo!?
                            Tired of spam? Yahoo! Mail has the best spam protection around
                            http://mail.yahoo.com
                          • bob_corrick
                            ... http://www.agiledata.org/essays/caseStudies.html Thank you. I downloaded the Cutter paper about a ThoughtWorks project for AOL broadband - taking an agile
                            Message 13 of 16 , Dec 18, 2006
                            • 0 Attachment
                              --- Scott Ambler <scottwambler@...> wrote:
                              > ... Case studies can be found from
                              http://www.agiledata.org/essays/caseStudies.html

                              Thank you.

                              I downloaded the Cutter paper about a ThoughtWorks project for AOL
                              broadband - taking an agile approach to the DBA role to improve the
                              build processes. This didn't mention refactoring at such a low level
                              of detail as "rename column".

                              I have a recent example of renaming a column (actually several
                              columns in several tables). This resulted in business-oriented
                              column names in the staging area of a data warehouse, instead of
                              the "legacy" names that we started with. What I did in each case -
                              not claiming any particular agility - was:
                              1. created a completely new staging table as a "select" of the
                              data from the table with the original column names, i.e. with no
                              data type changes.
                              2. reconciled the new table as the input to the next stage of the
                              ETL process (using O*acle W*rehouse B*ilder)
                              3. tested the ETL in a QA environment using SQL statements (of the
                              form <source> MINUS <target>, expecting no rows) on the way in and
                              on the way out of the new tables

                              This was fairly small scale: data was delayed for a day or so;
                              tables were tens of columns, tens of thousands of rows; fewer than
                              ten analysts were using the data.

                              Back to Scott's post:
                              > Here's a case study for you:
                              > Large retailer that needed to extend an existing web-based app
                              ...
                              > The QA folks said it was one of the most successful
                              > implementations that they'd ever seen.
                              ...
                              > - The business folks were very happy with us.
                              ... there follows a 'war story' (thank you) of success despite
                              conflicts between the project team (which had a project manager from
                              the client) and the client's own data group.

                              It would be interesting to hear about the present development
                              practices at that client. Did they stay with the agile approach? Are
                              the same people still working at and with the client? Are you able
                              to supply a URL to the client's web site? :-)

                              regards
                              Bob
                            • Scott Ambler
                              ... Various papers go to various levels of detail. Many case studies cover code refactoring, for example, yet rarely talk about Rename Method . ... This
                              Message 14 of 16 , Dec 18, 2006
                              • 0 Attachment
                                --- bob_corrick <bob.corrick@...> wrote:

                                > --- Scott Ambler <scottwambler@...> wrote:
                                > > ... Case studies can be found from
                                > http://www.agiledata.org/essays/caseStudies.html
                                >
                                > Thank you.
                                >
                                > I downloaded the Cutter paper about a ThoughtWorks
                                > project for AOL
                                > broadband - taking an agile approach to the DBA role
                                > to improve the
                                > build processes. This didn't mention refactoring at
                                > such a low level
                                > of detail as "rename column".

                                Various papers go to various levels of detail. Many
                                case studies cover code refactoring, for example, yet
                                rarely talk about "Rename Method".

                                >
                                > I have a recent example of renaming a column
                                > (actually several
                                > columns in several tables). This resulted in
                                > business-oriented
                                > column names in the staging area of a data
                                > warehouse, instead of
                                > the "legacy" names that we started with. What I did
                                > in each case -
                                > not claiming any particular agility - was:
                                > 1. created a completely new staging table as a
                                > "select" of the
                                > data from the table with the original column names,
                                > i.e. with no
                                > data type changes.
                                > 2. reconciled the new table as the input to the
                                > next stage of the
                                > ETL process (using O*acle W*rehouse B*ilder)
                                > 3. tested the ETL in a QA environment using SQL
                                > statements (of the
                                > form <source> MINUS <target>, expecting no rows) on
                                > the way in and
                                > on the way out of the new tables

                                This doesn't sound like renaming the columns to me,
                                but I could be wrong. Were the original source
                                columns actually renamed? From your description, all
                                it sounds like you did was an ETL and reconciliation
                                from several legacy sources into a DW. Important
                                stuff to do, mind you, but it's not refactoring.

                                > It would be interesting to hear about the present
                                > development
                                > practices at that client. Did they stay with the
                                > agile approach?

                                From what I know, I'm no longer there, they are still
                                on the path.

                                > Are
                                > the same people still working at and with the
                                > client?

                                The client is self-sustaining now (at least that's
                                what they claim, and I suspect that it's true). My
                                understanding is that after this project they took a
                                hard look at their DM group and realized that the DM
                                group wasn't fulfilling the promises that it was
                                making, didn't appear to be on any sort of plausible
                                track to do so, and didn't really seem to understand
                                what was actually needed by their organization. I
                                suspect that we're going to see more and more of this
                                as agile approaches are adopted by organizations --
                                it's really hard for the non-performers to hide within
                                agile environments, although it seems spectacularly
                                easy to do so within traditional environments. Time
                                will tell.


                                >Are you able
                                > to supply a URL to the client's web site? :-)

                                No. When I was consulting a standard clause in the
                                contract was a "No Names" paragraph and sometimes even
                                a "No Write Ups" clause. Most companies don't want
                                case studies written up.

                                If you keep your eye out at Cutter, they have a
                                tendency to publish good experience reports from time
                                to time. Whenever I hear about something regarding
                                agile database development I link to it from my case
                                studies page, but I'm sure that I don't find them all.

                                - Scott

                                Scott W. Ambler
                                Practice Leader Agile Development, IBM Methods Group
                                http://www-306.ibm.com/software/rational/bios/ambler.html

                                __________________________________________________
                                Do You Yahoo!?
                                Tired of spam? Yahoo! Mail has the best spam protection around
                                http://mail.yahoo.com
                              • bob_corrick
                                ... wrote: ... ... To clarify: yes, we renamed tens of columns in three staging tables of our data warehouse. You could call it a bulk refactoring of three ETL
                                Message 15 of 16 , Dec 18, 2006
                                • 0 Attachment
                                  --- In agileDatabases@yahoogroups.com, Scott Ambler <scottwambler@...>
                                  wrote:
                                  ...
                                  > This doesn't sound like renaming the columns to me,
                                  > but I could be wrong. Were the original source
                                  > columns actually renamed? From your description, all
                                  > it sounds like you did was an ETL and reconciliation
                                  > from several legacy sources into a DW. Important
                                  > stuff to do, mind you, but it's not refactoring.
                                  ...

                                  To clarify: yes, we renamed tens of columns in three staging tables of
                                  our data warehouse. You could call it a bulk refactoring of three ETL
                                  packages. Not as difficult as refactoring an operational database, I
                                  accept; just the closest I could get to providing some case study
                                  material, as invited.

                                  It was also interesting to hear about the client; I completely
                                  understand the confidentiality.

                                  Thank you for the prompt feedback,
                                  Bob
                                • Scott Ambler
                                  ... Cool. Thanks. ... It s one of the frustrating things about IT. If the project goes well the client thinks that if it gets written up they ll reveal
                                  Message 16 of 16 , Dec 18, 2006
                                  • 0 Attachment
                                    --- bob_corrick <bob.corrick@...> wrote:

                                    > --- In agileDatabases@yahoogroups.com, Scott Ambler
                                    > <scottwambler@...>
                                    > wrote:
                                    > ...
                                    > > This doesn't sound like renaming the columns to
                                    > me,
                                    > > but I could be wrong. Were the original source
                                    > > columns actually renamed? From your description,
                                    > all
                                    > > it sounds like you did was an ETL and
                                    > reconciliation
                                    > > from several legacy sources into a DW. Important
                                    > > stuff to do, mind you, but it's not refactoring.
                                    > ...
                                    >
                                    > To clarify: yes, we renamed tens of columns in three
                                    > staging tables of
                                    > our data warehouse. You could call it a bulk
                                    > refactoring of three ETL
                                    > packages. Not as difficult as refactoring an
                                    > operational database, I
                                    > accept; just the closest I could get to providing
                                    > some case study
                                    > material, as invited.

                                    Cool. Thanks.

                                    >
                                    > It was also interesting to hear about the client; I
                                    > completely
                                    > understand the confidentiality.

                                    It's one of the frustrating things about IT. If the
                                    project goes well the client thinks that if it gets
                                    written up they'll reveal competitive secrets to their
                                    competition, and if it doesn't go well they don't want
                                    to embarass themselves (OK, I can see that). Worse
                                    yet, when a case study is written up that reveals
                                    names, it's often devoid of information because of the
                                    first two issues.

                                    - Scott


                                    Scott W. Ambler
                                    Practice Leader Agile Development, IBM Methods Group
                                    http://www-306.ibm.com/software/rational/bios/ambler.html

                                    __________________________________________________
                                    Do You Yahoo!?
                                    Tired of spam? Yahoo! Mail has the best spam protection around
                                    http://mail.yahoo.com
                                  Your message has been successfully submitted and would be delivered to recipients shortly.