Loading ...
Sorry, an error occurred while loading the content.

Re: Some guidelines for database management

Expand Messages
  • thelazydba
    I totally agree with this approach and have successfully implemented such a scheme at a previous employer. My current environment presents me with some
    Message 1 of 13 , Nov 7, 2002
    • 0 Attachment
      I totally agree with this approach and have successfully implemented
      such a scheme at a previous employer. My current environment
      presents me with some additional challenges. Whereas the previous
      application was associated with a single Oracle schema, the current
      application spans multiple Oracle schemas. Previously, if I wanted
      to create another dev playground, or qa environment, or integration
      environment, I simply created another schema. Very simple, very easy.

      With the current environment, I would have to clone no less than four
      schemas to do the same. And then there is the issue of public
      synonyms and privileges, making the cloning approach very
      unattractive it not impossible. So, I'm thinking I have to create
      unique instances for every developer or environment I want to
      establish. Am I going down the right road?

      -thelazydba

      --- In agileDatabases@y..., "Ben Menasha" <bmenasha@b...> wrote:
      > Here are a number of guidelines that should be followed in order to
      successfully
      > manage changes to the database in a fast moving development effort.
      They where
      > tested on a recent project with great success.
      >
      > - Every developer needs control of a DBMS.
      > This is by far the most important guideline to follow. The
      ability to change
      > and reload the database independent of other developers is crucial
      for
      > developing new code, and not breaking existing development. If
      desktop machines
      > do not have enough power to run the database, then use a shared
      server to run
      > multiple instances of a DBMS.
      >
      >
      > - Database changes must be tracked.
      > Every change to the database done in a release needs to be
      checked into a
      > version control system (vcs). If a change is done with a GUI tool,
      make sure the
      > change is scripted and checked into the vcs.
      >
      > - Database creation/loading must to automated.
      > Since the database might need to be recreated and loaded on each
      checkout from
      > the vcs, it must be able to be performed fast and without human
      intervention.
      > This is often done by invoking vendor specific command line tools.
      First the
      > create table scripts next, each release patch is applied in turn to
      bring the
      > database up to the current rev.
      >
      > - Everyone owns the database.
      > Just like code, no one person is responsible for the health of
      the database.
      > Everyone needs to take ownership of every table. Often Database
      Administrators
      > will review the schema and tune queries for performance. This is no
      different
      > then any other code review. DBAs should never gatekeeper changes,
      everyone
      > should be able to make a change to the scripts in the vcs at any
      time.
      >
      > - Test data should be as close to production as possible.
      > For very large databases it is not possible to check all the data
      into a vcs.
      > However it will be possible to check part of it in. This test data
      will need to
      > be kept in sync with what is in production, and will need be
      managed in the vcs.
      > When a checkout and database rebuild is performed, the test data
      should be
      > loaded.
      >
      > What follows is an example of the path structure that was used on
      the recent
      > project.
      >
      > /projectname
      > /source
      > ....
      > /data -- a subset of exported data
      > /sql
      > tables.sql -- create table scripts
      > ....
      > /patch --patches to bring the db up to a release
      > /1.0
      > patch.sql
      > /1.1
      > patch.sql
      >
      >
      >
      > When the database is rebuild, first the initial create table
      scripts where run
      > in the sql directory. The patch scripts in projectname/sq/patch/1.0
      where run
      > next. At this point in time, the database schema should match
      exactly with what
      > is in production. The test data is loaded from the data directory,
      this required
      > care in loading the data in the proper order to ensure integrity
      constraints are
      > meet. Finally the sql scripts in patch 1.1 are applied.
      >
      > When release 1.1 is finished and ready to be deployed, the patch
      scripts need to
      > be run against the staging environment to ensure that they work
      against the full
      > dataset that is in production. This is because the vcs might only
      contain a
      > subset of the production database (because of size constraints).
      Once the
      > release manager is confident in there correctness, the release can
      be pushed to
      > production. A new subset of the production data can be cropped from
      the db, and
      > checked into the vcs, and the test data will now only be loaded
      after the
      > patches from the 1.1 release directory have been run, and a new 1.2
      patch
      > directory is created.
      >
      > Future changes to the database are only made to the most recent
      patch directory.
      > The table creation scripts should not typically be changed after
      the initial
      > release to production has been made. This is because the will only
      be run in
      > production once, from then on, only patch scripts will be run.
      >
      > By grouping database changes by release, it is possible to upgrade
      a database
      > from a previous release to the next. Every change made to the
      database will be
      > applied to that releases patch script. In fact when a checkout is
      performed all
      > these patches will be run, so they will get exercised from day one.
      >
      > You need to tailor such a scheme to suite your needs, for example
      if you can
      > completely drop and repopulate your production database, then you
      can do away
      > with the whole patches directories. Or you database might be small
      enough to
      > manage in a vcs. However, the guidelines above seem applicable to
      everyone.
      >
      > hope this was worth your time.
      > -b
    • psadalage
      I would agree that cloning the instance would be a better idea in this case..you should script the whole process from creating the instance, create the
      Message 2 of 13 , Nov 7, 2002
      • 0 Attachment
        I would agree that cloning the instance would be a better idea in
        this case..you should script the whole process from creating the
        instance, create the database, tablespace, users, tables and data.

        I would also try to investigate why would it not be possible to fold
        the multiple schemas into a single schema.
        Pramod

        --- In agileDatabases@y..., "thelazydba" <thelazydba@y...> wrote:
        > I totally agree with this approach and have successfully
        implemented
        > such a scheme at a previous employer. My current environment
        > presents me with some additional challenges. Whereas the previous
        > application was associated with a single Oracle schema, the current
        > application spans multiple Oracle schemas. Previously, if I wanted
        > to create another dev playground, or qa environment, or integration
        > environment, I simply created another schema. Very simple, very
        easy.
        >
        > Wit
        h the current environment, I would have to clone no less than four
        > schemas to do the same. And then there is the issue of public
        > synonyms and privileges, making the cloning approach very
        > unattractive it not impossible. So, I'm thinking I have to create
        > unique instances for every developer or environment I want to
        > establish. Am I going down the right road?
        >
        > -thelazydba
        >
        > --- In agileDatabases@y..., "Ben Menasha" <bmenasha@b...> wrote:
        > > Here are a number of guidelines that should be followed in order
        to
        > successfully
        > > manage changes to the database in a fast moving development
        effort.
        > They where
        > > tested on a recent project with great success.
        > >
        > > - Every developer needs control of a DBMS.
        > > This is by far the most important guideline to follow. The
        > ability to change
        > > and reload the database independent of other developers is
        crucial
        > for
        > > developing new code, and not breaking existing development. If
        > desktop machines
        > > do not have enough power to run the database, then use a shared
        > server to run
        > > multiple instances of a DBMS.
        > >
        > >
        > > - Database changes must be tracked.
        > > Every change to the database done in a release needs to be
        > checked into a
        > > version control system (vcs). If a change is done with a GUI
        tool,
        > make sure the
        > > change is scripted and checked into the vcs.
        > >
        > > - Database creation/loading must to automated.
        > > Since the database might need to be recreated and loaded on
        each
        > checkout from
        > > the vcs, it must be able to be performed fast and without human
        > intervention.
        > > This is often done by invoking vendor specific command line
        tools.
        > First the
        > > create table scripts next, each release patch is applied in turn
        to
        > bring the
        > > database up to the current rev.
        > >
        > > - Everyone owns the database.
        > > Just like code, no one person is responsible for the health of
        > the database.
        > > Everyone needs to take ownership of every table. Often Database
        > Administrators
        > > will review the schema and tune queries for performance. This is
        no
        > different
        > > then any other code review. DBAs should never gatekeeper changes,
        > everyone
        > > should be able to make a change to the scripts in the vcs at any
        > time.
        > >
        > > - Test data should be as close to production as possible.
        > > For very large databases it is not possible to check all the
        data
        > into a vcs.
        > > However it will be possible to check part of it in. This test
        data
        > will need to
        > > be kept in sync with what is in production, and will need be
        > managed in the vcs.
        > > When a checkout and database rebuild is performed, the test data
        > should be
        > > loaded.
        > >
        > > What follows is an example of the path structure that was used on
        > the recent
        > > project.
        > >
        > > /projectname
        > > /source
        > > ....
        > > /data -- a subset of exported data
        > > /sql
        > > tables.sql -- create table scripts
        > > ....
        > > /patch --patches to bring the db up to a release
        > > /1.0
        > > patch.sql
        > > /1.1
        > > patch.sql
        > >
        > >
        > >
        > > When the database is rebuild, first the initial create table
        > scripts where run
        > > in the sql directory. The patch scripts in
        projectname/sq/patch/1.0
        > where run
        > > next. At this point in time, the database schema should match
        > exactly with what
        > > is in production. The test data is loaded from the data
        directory,
        > this required
        > > care in loading the data in the proper order to ensure integrity
        > constraints are
        > > meet. Finally the sql scripts in patch 1.1 are applied.
        > >
        > > When release 1.1 is finished and ready to be deployed, the patch
        > scripts need to
        > > be run against the staging environment to ensure that they work
        > against the full
        > > dataset that is in production. This is because the vcs might only
        > contain a
        > > subset of the production database (because of size constraints).
        > Once the
        > > release manager is confident in there correctness, the release
        can
        > be pushed to
        > > production. A new subset of the production data can be cropped
        from
        > the db, and
        > > checked into the vcs, and the test data will now only be loaded
        > after the
        > > patches from the 1.1 release directory have been run, and a new
        1.2
        > patch
        > > directory is created.
        > >
        > > Future changes to the database are only made to the most recent
        > patch directory.
        > > The table creation scripts should not typically be changed after
        > the initial
        > > release to production has been made. This is because the will
        only
        > be run in
        > > production once, from then on, only patch scripts will be run.
        > >
        > > By grouping database changes by release, it is possible to
        upgrade
        > a database
        > > from a previous release to the next. Every change made to the
        > database will be
        > > applied to that releases patch script. In fact when a checkout is
        > performed all
        > > these patches will be run, so they will get exercised from day
        one.
        > >
        > > You need to tailor such a scheme to suite your needs, for example
        > if you can
        > > completely drop and repopulate your production database, then you
        > can do away
        > > with the whole patches directories. Or you database might be
        small
        > enough to
        > > manage in a vcs. However, the guidelines above seem applicable to
        > everyone.
        > >
        > > hope this was worth your time.
        > > -b
      • Bayley, Alistair
        ... If the environment is like mine then developers have referenced objects in specific schemas i.e. object names have been prefixed by schema names. I have
        Message 3 of 13 , Nov 7, 2002
        • 0 Attachment
          > I would also try to investigate why would it not be possible to fold
          > the multiple schemas into a single schema.

          If the environment is like mine then developers have referenced objects in
          specific schemas i.e. object names have been prefixed by schema names. I
          have just gone through our entire source code repository and removed schema
          references, so that objects can be loaded into a schema other than the
          "live" schema.

          Of course the other reason may be name collisions e.g. a person table may
          exist in more than one schema.

          I suggested the idea of every developer having their own database here and
          the other developers poo-poo'ed the idea. The primary objection appeared to
          be "we're not DBAs, we don't want to administer our own database".


          -----Original Message-----
          From: psadalage [mailto:psadalage@...]
          Sent: 07 November 2002 16:56
          To: agileDatabases@yahoogroups.com
          Subject: [agileDatabases] Re: Some guidelines for database management


          I would agree that cloning the instance would be a better idea in
          this case..you should script the whole process from creating the
          instance, create the database, tablespace, users, tables and data.

          I would also try to investigate why would it not be possible to fold
          the multiple schemas into a single schema.
          Pramod

          --- In agileDatabases@y..., "thelazydba" <thelazydba@y...> wrote:
          > I totally agree with this approach and have successfully implemented
          > such a scheme at a previous employer. My current environment
          > presents me with some additional challenges. Whereas the previous
          > application was associated with a single Oracle schema, the current
          > application spans multiple Oracle schemas. Previously, if I wanted
          > to create another dev playground, or qa environment, or integration
          > environment, I simply created another schema. Very simple, very easy.
          >
          > With the current environment, I would have to clone no less than four
          > schemas to do the same. And then there is the issue of public
          > synonyms and privileges, making the cloning approach very
          > unattractive it not impossible. So, I'm thinking I have to create
          > unique instances for every developer or environment I want to
          > establish. Am I going down the right road?
          >
          > -thelazydba


          *****************************************************************
          The information in this email and in any attachments is
          confidential and intended solely for the attention and use
          of the named addressee(s). This information may be
          subject to legal professional or other privilege or may
          otherwise be protected by work product immunity or other
          legal rules. It must not be disclosed to any person without
          our authority.

          If you are not the intended recipient, or a person
          responsible for delivering it to the intended recipient, you
          are not authorised to and must not disclose, copy,
          distribute, or retain this message or any part of it.
          *****************************************************************
        • Rover Still
          ... Another reason is that database may behave differently from a personal database to a database with multiple concurrent users. Issues like locking,
          Message 4 of 13 , Nov 8, 2002
          • 0 Attachment
            > I suggested the idea of every developer having their
            > own database here and
            > the other developers poo-poo'ed the idea. The
            > primary objection appeared to
            > be "we're not DBAs, we don't want to administer our
            > own database".

            Another reason is that database may behave differently
            from a "personal database" to a database with multiple
            concurrent users. Issues like locking, blocking, dead
            locking don't happen to your personal database WILL
            happen in production. Worse, you may get inconsistant
            result if other users modified your data during
            executing of your program.

            If you can not handle other developers modify your
            data, there is a great chance that the program work
            perfectly in your personal database may not work in
            real production environment.


            __________________________________________________
            Do you Yahoo!?
            U2 on LAUNCH - Exclusive greatest hits videos
            http://launch.yahoo.com/u2
          • Jeffrey D. Brekke
            Wouldn t you have a better chance of testing these issues in a personal db, than just having everyone working in the same db and hoping you ll hit these
            Message 5 of 13 , Nov 8, 2002
            • 0 Attachment
              Wouldn't you have a better chance of testing these issues in a personal
              db, than just having everyone working in the same db and hoping you'll hit
              these issues? If locking, blocking, and deadlocking are concerns, you
              would write tests for them. The tests will run and verify behavior in
              your personal db as well as the testing db, etc.

              On Fri, 8 Nov 2002, Rover Still wrote:

              > > I suggested the idea of every developer having their
              > > own database here and
              > > the other developers poo-poo'ed the idea. The
              > > primary objection appeared to
              > > be "we're not DBAs, we don't want to administer our
              > > own database".
              >
              > Another reason is that database may behave differently
              > from a "personal database" to a database with multiple
              > concurrent users. Issues like locking, blocking, dead
              > locking don't happen to your personal database WILL
              > happen in production. Worse, you may get inconsistant
              > result if other users modified your data during
              > executing of your program.
              >
              > If you can not handle other developers modify your
              > data, there is a great chance that the program work
              > perfectly in your personal database may not work in
              > real production environment.
              >
              >
              > __________________________________________________
              > Do you Yahoo!?
              > U2 on LAUNCH - Exclusive greatest hits videos
              > http://launch.yahoo.com/u2
              >
              >
              > To unsubscribe from this group, send an email to:
              > agileDatabases-unsubscribe@yahoogroups.com
              >
              >
              >
              > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
              >
              >

              --
              =====================================================================
              Jeffrey D. Brekke jbrekke@...
              Software Engineer brekke@...
              Wisconsin, USA ekkerbj@...
            • Bayley, Alistair
              I agree. The personal DB (or schema) is for unit testing. If your tests require concurrent updates, deadlocks, etc, then you should create them to do just that
              Message 6 of 13 , Nov 8, 2002
              • 0 Attachment
                I agree. The personal DB (or schema) is for unit testing. If your tests
                require concurrent updates, deadlocks, etc, then you should create them to
                do just that in your own database, without requiring (unpredictable)
                intervention from other users. The point is that you have total control over
                the database, and you can perform your testing in a known environment.

                > If you can not handle other developers modify your
                > data, there is a great chance that the program work
                > perfectly in your personal database may not work in
                > real production environment.

                It's not just that I can't handle other developer's modifying my data, it's
                also that they do not want me trampling on theirs. What do I do if I want to
                test a boundary case, where a table is empty? Do I delete the shared data?
                (I think not.)

                You should still have a database (probably shared, recopied regularly) that
                is a copy of the production database (or as close as you can afford) for
                volume/concurrency testing (and for practising releasing to).


                -----Original Message-----
                From: Jeffrey D. Brekke [mailto:jbrekke@...]
                Sent: 08 November 2002 15:35
                To: agileDatabases@yahoogroups.com
                Subject: RE: [agileDatabases] Re: Some guidelines for database
                management


                Wouldn't you have a better chance of testing these issues in a personal
                db, than just having everyone working in the same db and hoping you'll hit
                these issues? If locking, blocking, and deadlocking are concerns, you
                would write tests for them. The tests will run and verify behavior in
                your personal db as well as the testing db, etc.

                On Fri, 8 Nov 2002, Rover Still wrote:

                > > I suggested the idea of every developer having their
                > > own database here and
                > > the other developers poo-poo'ed the idea. The
                > > primary objection appeared to
                > > be "we're not DBAs, we don't want to administer our
                > > own database".
                >
                > Another reason is that database may behave differently
                > from a "personal database" to a database with multiple
                > concurrent users. Issues like locking, blocking, dead
                > locking don't happen to your personal database WILL
                > happen in production. Worse, you may get inconsistant
                > result if other users modified your data during
                > executing of your program.
                >
                > If you can not handle other developers modify your
                > data, there is a great chance that the program work
                > perfectly in your personal database may not work in
                > real production environment.


                *****************************************************************
                The information in this email and in any attachments is
                confidential and intended solely for the attention and use
                of the named addressee(s). This information may be
                subject to legal professional or other privilege or may
                otherwise be protected by work product immunity or other
                legal rules. It must not be disclosed to any person without
                our authority.

                If you are not the intended recipient, or a person
                responsible for delivering it to the intended recipient, you
                are not authorised to and must not disclose, copy,
                distribute, or retain this message or any part of it.
                *****************************************************************
              • Ben Menasha
                From: Rover Still ... database WILL happen in production. As you note, going straight from development to production is guaranteed
                Message 7 of 13 , Nov 8, 2002
                • 0 Attachment
                  From: "Rover Still" <rover_still@...>
                  >Issues like locking, blocking, dead locking don't happen to your personal
                  database WILL happen in production.

                  As you note, going straight from development to production is guaranteed to
                  generate problems such as this. Which is why a testing environment and
                  staging environment is critical. Developers need access to both of these
                  environments to gather information. Without a staging environment you will
                  end up "playing" with production to diagnose problems, and without a testing
                  environment, you will not be able to test during development in order to
                  catch problems early.

                  >Bayley, Alistair Writes
                  >. The primary objection appeared to be "we're not DBAs, we don't want to
                  administer our own database".

                  This is not a way to offload a DBA's work onto developers, but a strategy to
                  facilitate fast iterative development which will most likely cause more work
                  for the DBA! Perhaps they did not understand what was being asked of them?
                  They will need to put work into integrating these scripts into the build
                  process. But this is a one time charge, they will not be taking on any more
                  responsibilities over the long haul, (unless they are not managing database
                  changes in a version control system now!). It is still the DBA who will
                  create the scripts to manage schemas/instances, review patches and scripts..
                  etc.

                  -b





                  ----- Original Message -----
                  From: "Rover Still" <rover_still@...>
                  To: <agileDatabases@yahoogroups.com>
                  Sent: Friday, November 08, 2002 10:23 AM
                  Subject: RE: [agileDatabases] Re: Some guidelines for database management


                  > > I suggested the idea of every developer having their
                  > > own database here and
                  > > the other developers poo-poo'ed the idea. The
                  > > primary objection appeared to
                  > > be "we're not DBAs, we don't want to administer our
                  > > own database".
                  >
                  > Another reason is that database may behave differently
                  > from a "personal database" to a database with multiple
                  > concurrent users. Issues like locking, blocking, dead
                  > locking don't happen to your personal database WILL
                  > happen in production. Worse, you may get inconsistant
                  > result if other users modified your data during
                  > executing of your program.
                  >
                  > If you can not handle other developers modify your
                  > data, there is a great chance that the program work
                  > perfectly in your personal database may not work in
                  > real production environment.
                  >
                  >
                  > __________________________________________________
                  > Do you Yahoo!?
                  > U2 on LAUNCH - Exclusive greatest hits videos
                  > http://launch.yahoo.com/u2
                  >
                  >
                  > To unsubscribe from this group, send an email to:
                  > agileDatabases-unsubscribe@yahoogroups.com
                  >
                  >
                  >
                  > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                  >
                  >
                • Rover Still
                  ... Well, I d love to have BOTH personal and staging environment available to developers. That s luxury. But you still need to take database concurrent access
                  Message 8 of 13 , Nov 8, 2002
                  • 0 Attachment
                    > Which is why a testing environment and
                    > staging environment is critical. Developers need
                    > access to both of these environments to gather
                    > information. Without a staging environment you
                    > will end up "playing" with production to diagnose
                    > problems, and without a testing environment, you
                    > will not be able to test during
                    > development in order to catch problems early.

                    Well, I'd love to have BOTH personal and staging
                    environment available to developers. That's luxury.

                    But you still need to take database concurrent access
                    consideration into very early stage. Wait till after
                    finishing unit test to tackle concurrency problem is
                    usually too late. It may cost you to rewrite a large
                    chunk of code.

                    In fact, if you have concurrency consideration from
                    the beginning, you will find sharing a development
                    database with others is not that annoying. Sometimes,
                    it is actually an indication that your code has
                    concurrency problem when you found other developers
                    are disturbing you.



                    __________________________________________________
                    Do you Yahoo!?
                    U2 on LAUNCH - Exclusive greatest hits videos
                    http://launch.yahoo.com/u2
                  • psadalage
                    ... You can give them control of schemas which are part of the database or instance maintained by the DBA. This database machine is the central machine so the
                    Message 9 of 13 , Nov 8, 2002
                    • 0 Attachment
                      >I suggested the idea of every developer having their own database
                      >here and
                      >the other developers poo-poo'ed the idea. The primary objection
                      >appeared to
                      >be "we're not DBAs, we don't want to administer our own database".

                      You can give them control of schemas which are part of the database
                      or instance maintained by the DBA. This database machine is the
                      central machine so the developers don't have to be DBA's. Think of it
                      this way..would the developers really want to work with one
                      application instance combined among all of them. I don't think soo,
                      because it will slow them down.. the same logic when applied to
                      database (or schema to be precise) will make them more productive.

                      Pramod


                      --- In agileDatabases@y..., "Bayley, Alistair" <alistair_bayley@l...>
                      wrote:
                      > > I would also try to investigate why would it not be possible to
                      fold
                      > > the multiple schemas into a single schema.
                      >
                      > If the environment is like mine then developers have referenced
                      objects in
                      > specific schemas i.e. object names have been prefixed by schema
                      names. I
                      > have just gone through our entire source code repository and
                      removed schema
                      > references, so that objects can be loaded into a schema other than
                      the
                      > "live" schema.
                      >
                      > Of course the other reason may be name collisions e.g. a person
                      table may
                      > exist in more than one schema.
                      >
                      > I suggested the idea of every developer having their own database
                      here and
                      > the other developers poo-poo'ed the idea. The primary objection
                      appeared to
                      > be "we're not DBAs, we don't want to administer our own database".
                      >
                      >
                      > -----Original Message-----
                      > From: psadalage [mailto:psadalage@y...]
                      > Sent: 07 November 2002 16:56
                      > To: agileDatabases@y...
                      > Subject: [agileDatabases] Re: Some guidelines for database
                      management
                      >
                      >
                      > I would agree that cloning the instance would be a better idea in
                      > this case..you should script the whole process from creating the
                      > instance, create the database, tablespace, users, tables and data.
                      >
                      > I would also try to investigate why would it not be possible to
                      fold
                      > the multiple schemas into a single schema.
                      > Pramod
                      >
                      > --- In agileDatabases@y..., "thelazydba" <thelazydba@y...> wrote:
                      > > I totally agree with this approach and have successfully
                      implemented
                      > > such a scheme at a previous employer. My current environment
                      > > presents me with some additional challenges. Whereas the
                      previous
                      > > application was associated with a single Oracle schema, the
                      current
                      > > application spans multiple Oracle schemas. Previously, if I
                      wanted
                      > > to create another dev playground, or qa environment, or
                      integration
                      > > environment, I simply created another schema. Very simple, very
                      easy.
                      > >
                      > > With the current environment, I would have to clone no less than
                      four
                      > > schemas to do the same. And then there is the issue of public
                      > > synonyms and privileges, making the cloning approach very
                      > > unattractive it not impossible. So, I'm thinking I have to
                      create
                      > > unique instances for every developer or environment I want to
                      > > establish. Am I going down the right road?
                      > >
                      > > -thelazydba
                      >
                      >
                      > *****************************************************************
                      > The information in this email and in any attachments is
                      > confidential and intended solely for the attention and use
                      > of the named addressee(s). This information may be
                      > subject to legal professional or other privilege or may
                      > otherwise be protected by work product immunity or other
                      > legal rules. It must not be disclosed to any person without
                      > our authority.
                      >
                      > If you are not the intended recipient, or a person
                      > responsible for delivering it to the intended recipient, you
                      > are not authorised to and must not disclose, copy,
                      > distribute, or retain this message or any part of it.
                      > *****************************************************************
                    • psadalage
                      you would never go from development to production would you.. also HOPING that developers while working on the same database MAY encounter concurrency bugs, is
                      Message 10 of 13 , Nov 8, 2002
                      • 0 Attachment
                        you would never go from development to production would you.. also
                        HOPING that developers while working on the same database MAY
                        encounter concurrency bugs, is a waste of time. What if the
                        developers never hit a concurrency bug, we can't assume that the app
                        is okay with concurrency control. It is better to have a test suite
                        that tests for these specific issues.

                        Pramod

                        --- In agileDatabases@y..., Rover Still <rover_still@y...> wrote:
                        > > I suggested the idea of every developer having their
                        > > own database here and
                        > > the other developers poo-poo'ed the idea. The
                        > > primary objection appeared to
                        > > be "we're not DBAs, we don't want to administer our
                        > > own database".
                        >
                        > Another reason is that database may behave differently
                        > from a "personal database" to a database with multiple
                        > concurrent users. Issues like locking, blocking, dead
                        > locking don't happen to your personal database WILL
                        > happen in production. Worse, you may get inconsistant
                        > result if other users modified your data during
                        > executing of your program.
                        >
                        > If you can not handle other developers modify your
                        > data, there is a great chance that the program work
                        > perfectly in your personal database may not work in
                        > real production environment.
                        >
                        >
                        > __________________________________________________
                        > Do you Yahoo!?
                        > U2 on LAUNCH - Exclusive greatest hits videos
                        > http://launch.yahoo.com/u2
                      • Kent Beck
                        If you have a staging server, there will always be a floor on how short you can make the cycle between concept and deployment, and there will always be a
                        Message 11 of 13 , Nov 11, 2002
                        • 0 Attachment
                          Message
                          If you have a staging server, there will always be a floor on how short you can make the cycle between concept and deployment, and there will always be a challenge keeping the staging and production environments identical. It would be better to have immediate feedback on the developers' machines. Until this is possible, use the staging server, but recognize it as an intermediate setup.
                           
                          Kent
                          -----Original Message-----
                          From: sentto-7758546-20-1036773228-kent=threeriversinstitute.org@... [mailto:sentto-7758546-20-1036773228-kent=threeriversinstitute.org@...] On Behalf Of Ben Menasha
                          Sent: Friday, November 08, 2002 8:34 AM
                          To: agileDatabases@yahoogroups.com
                          Subject: Re: [agileDatabases] Re: Some guidelines for database management

                          From: "Rover Still" <rover_still@...>
                          >Issues like locking, blocking, dead locking don't happen to your personal
                          database WILL  happen in production.

                          As you note, going  straight from development to production is guaranteed to
                          generate problems such as this. Which is why a testing environment and
                          staging environment is critical.  Developers need access to both of these
                          environments to gather information. Without a staging environment you will
                          end up "playing" with production to diagnose problems, and without a testing
                          environment, you will not be able to test during development in order to
                          catch problems early.

                          >Bayley, Alistair Writes
                          >. The primary objection appeared to  be "we're not DBAs, we don't want to
                          administer our  own database".

                          This is not a way to offload a DBA's work onto developers, but a strategy to
                          facilitate fast iterative development which will most likely cause more work
                          for the DBA!  Perhaps they did not understand what was being asked of them?
                          They will need to put work into integrating these scripts into the build
                          process. But this is a one time charge, they will not be taking on any more
                          responsibilities over the long haul, (unless they are not managing database
                          changes in a version control system now!).  It is still the DBA who will
                          create the scripts to manage schemas/instances, review patches and scripts..
                          etc.

                          -b





                          ----- Original Message -----
                          From: "Rover Still" <rover_still@...>
                          To: <agileDatabases@yahoogroups.com>
                          Sent: Friday, November 08, 2002 10:23 AM
                          Subject: RE: [agileDatabases] Re: Some guidelines for database management


                          > > I suggested the idea of every developer having their
                          > > own database here and
                          > > the other developers poo-poo'ed the idea. The
                          > > primary objection appeared to
                          > > be "we're not DBAs, we don't want to administer our
                          > > own database".
                          >
                          > Another reason is that database may behave differently
                          > from a "personal database" to a database with multiple
                          > concurrent users. Issues like locking, blocking, dead
                          > locking don't happen to your personal database WILL
                          > happen in production. Worse, you may get inconsistant
                          > result if other users modified your data during
                          > executing of your program.
                          >
                          > If you can not handle other developers modify your
                          > data, there is a great chance that the program work
                          > perfectly in your personal database may not work in
                          > real production environment.
                          >
                          >
                          > __________________________________________________
                          > Do you Yahoo!?
                          > U2 on LAUNCH - Exclusive greatest hits videos
                          > http://launch.yahoo.com/u2
                          >
                          >
                          > To unsubscribe from this group, send an email to:
                          > agileDatabases-unsubscribe@yahoogroups.com
                          >
                          >
                          >
                          > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                          >
                          >



                          To unsubscribe from this group, send an email to:
                          agileDatabases-unsubscribe@yahoogroups.com



                          Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
                        • patrickdlogan
                          ... Sorry I have not caught up with all the messages yet. While I do catch up I ll also throw out a few thoughts. Forgive any duplication. * Normalization is
                          Message 12 of 13 , Nov 15, 2002
                          • 0 Attachment
                            --- In agileDatabases@y..., "Ben Menasha" <bmenasha@b...> wrote:
                            > Here are a number of guidelines...

                            Sorry I have not caught up with all the messages yet. While I do catch
                            up I'll also throw out a few thoughts. Forgive any duplication.

                            * Normalization is not all its cracked up to be. Star schemas have
                            fully 3NF fact tables and 2NF or less dimension tables. Queries are
                            easy to write and efficient to run.

                            * On the OLTP side many situations would support writing directly to
                            a star schema model rather than a fully normalized model.
                            Application
                            servers can enforce all the rules you'd otherwise try to put into
                            the database's integrity features, and performance is often not an
                            issue.

                            * Star schema models are easier to refactor *because* they are less
                            normalized. They also map more easily to an OO runtime model,
                            especially "adaptive object models" that can build their
                            representations from the DB meta data.

                            * It's better to add a column of pre-calculated values than to perform
                            calculations at run-time. Queries are simplified. For example
                            calendar calculations are much easier when the calendar table is in
                            a dimension table at the day level and each row has a "number of
                            days since time began" column. Then there is no fidgiting with month
                            boundaries, etc. Make dimension tables very wide with helpful
                            additional columns as you find the need for them. Columns are easy
                            to add.

                            * Data quality is almost always a huge issue, more so when combining
                            information from multiple sources. Build quality information ("meta
                            data") directly into the tables... where, when, how
                            good. Understanding how good your data is will help you be agile
                            with that data.

                            -Patrick
                          Your message has been successfully submitted and would be delivered to recipients shortly.