Loading ...
Sorry, an error occurred while loading the content.

Behavior Driven Database Development

Expand Messages
  • ginitram
    The Methods & Tools newsletter has released in its html archive section the article Behavior Driven Database Development by Pramod Sadalage. This article
    Message 1 of 7 , Jan 5, 2009
    • 0 Attachment
      The Methods & Tools newsletter has released in its html archive section
      the article "Behavior Driven Database Development" by Pramod Sadalage.

      This article shows how the behavior driven development can be applied
      to database development and how it can be used to develop and design
      databases in an iterative and incremental way.

      http://www.methodsandtools.com/archive/archive.php?id=78
    • Gabriel Tanase
      Nice little article, however I can t resist a small complaint. It doesn t make any reference to and makes no explicit provision for multi-user access issues,
      Message 2 of 7 , Jan 6, 2009
      • 0 Attachment
        Nice little article, however I can't resist a small complaint.
        It doesn't make any reference to and makes no explicit provision for
        multi-user access issues, which IMHO is a must in any non-trivial db-based
        development.

        For example, the test shouldAssignPrimaryKeyValuesFromSequence()should be
        designed to be an "atomic transaction", e.g. as below (apologies for the
        bastardized syntax; I'm not a Java programmer):

        @Test
        public void shouldAssignPrimaryKeyValuesFromSequence() {
        *new lock_flag = Lock_Movie_PK_Context();*
        Movie movie = createAndSave("PKASSIGNED");
        Long currentPKValue = getCurrentValueForSequence("S_MOVIE");
        *lock_flag = Unlock_Movie_PK_Context();*
        assertNotNull("Movie should have been assigned a ID", movie.getId());
        assertEquals("ID should have been same",movie.getId(),currentPKValue);


        The problem is that in a multi-user environment the movie id value created
        by
        Movie movie = createAndSave("PKASSIGNED");
        and stored into the movie object may be different from the one then
        retrieved by
        Long currentPKValue = getCurrentValueForSequence("S_MOVIE");
        since another user may have created another movie, hence may have already
        used a new sequence value, between the two statements.
        Therefore the test would appear to fail (occasionally) purely because it is
        executed in a multi-user environment.


        Regards,
        Gabriel

        2009/1/5 ginitram <ginitram@...>

        > The Methods & Tools newsletter has released in its html archive section
        > the article "Behavior Driven Database Development" by Pramod Sadalage.
        >
        > This article shows how the behavior driven development can be applied
        > to database development and how it can be used to develop and design
        > databases in an iterative and incremental way.
        >
        > http://www.methodsandtools.com/archive/archive.php?id=78
        >
        > .
        >
        >
        >


        [Non-text portions of this message have been removed]
      • psadalage
        Hello Gabriel, I think there is confusion between Production environment and Unit testing/continuous integration environment. Mostly in all the projects I have
        Message 3 of 7 , Jan 6, 2009
        • 0 Attachment
          Hello Gabriel,

          I think there is confusion between Production environment and Unit
          testing/continuous integration environment.

          Mostly in all the projects I have worked on each developer (pair) has
          a database/schema dedicated to himself (herself). In the test
          specification referred below, the intent is to check that we are
          assigning PK values from the correct sequence and that the sequence
          increments in the correct increments.

          if multiple users are running unit tests against the same schema, you
          may run in many other problems

          Hope that helps

          Pramod

          --- In agileDatabases@yahoogroups.com, "Gabriel Tanase"
          <gabtanase@...> wrote:
          >
          > Nice little article, however I can't resist a small complaint.
          > It doesn't make any reference to and makes no explicit provision for
          > multi-user access issues, which IMHO is a must in any non-trivial
          db-based
          > development.
          >
          > For example, the test
          shouldAssignPrimaryKeyValuesFromSequence()should be
          > designed to be an "atomic transaction", e.g. as below (apologies for the
          > bastardized syntax; I'm not a Java programmer):
          >
          > @Test
          > public void shouldAssignPrimaryKeyValuesFromSequence() {
          > *new lock_flag = Lock_Movie_PK_Context();*
          > Movie movie = createAndSave("PKASSIGNED");
          > Long currentPKValue = getCurrentValueForSequence("S_MOVIE");
          > *lock_flag = Unlock_Movie_PK_Context();*
          > assertNotNull("Movie should have been assigned a ID", movie.getId());
          > assertEquals("ID should have been same",movie.getId(),currentPKValue);
          >
          >
          > The problem is that in a multi-user environment the movie id value
          created
          > by
          > Movie movie = createAndSave("PKASSIGNED");
          > and stored into the movie object may be different from the one then
          > retrieved by
          > Long currentPKValue = getCurrentValueForSequence("S_MOVIE");
          > since another user may have created another movie, hence may have
          already
          > used a new sequence value, between the two statements.
          > Therefore the test would appear to fail (occasionally) purely
          because it is
          > executed in a multi-user environment.
          >
          >
          > Regards,
          > Gabriel
          >
          > 2009/1/5 ginitram <ginitram@...>
          >
          > > The Methods & Tools newsletter has released in its html archive
          section
          > > the article "Behavior Driven Database Development" by Pramod Sadalage.
          > >
          > > This article shows how the behavior driven development can be applied
          > > to database development and how it can be used to develop and design
          > > databases in an iterative and incremental way.
          > >
          > > http://www.methodsandtools.com/archive/archive.php?id=78
          > >
          > > .
          > >
          > >
          > >
          >
          >
          > [Non-text portions of this message have been removed]
          >
        • Max Guernsey, III
          Should not the tests be responsible for creating their own database instances? Max Guernsey, III Managing Member, Hexagon Software
          Message 4 of 7 , Jan 6, 2009
          • 0 Attachment
            Should not the tests be responsible for creating their own database
            instances?



            Max Guernsey, III

            Managing Member, Hexagon Software

            <http://www.hexsw.com/> http://www.hexsw.com

            <http://www.dataconstructor.com/> http://www.dataconstructor.com



            From: agileDatabases@yahoogroups.com [mailto:agileDatabases@yahoogroups.com]
            On Behalf Of psadalage
            Sent: Tuesday, January 06, 2009 9:54 AM
            To: agileDatabases@yahoogroups.com
            Subject: [agileDatabases] Re: Behavior Driven Database Development



            Hello Gabriel,

            I think there is confusion between Production environment and Unit
            testing/continuous integration environment.

            Mostly in all the projects I have worked on each developer (pair) has
            a database/schema dedicated to himself (herself). In the test
            specification referred below, the intent is to check that we are
            assigning PK values from the correct sequence and that the sequence
            increments in the correct increments.

            if multiple users are running unit tests against the same schema, you
            may run in many other problems

            Hope that helps

            Pramod

            --- In agileDatabases@yahoogroups.com
            <mailto:agileDatabases%40yahoogroups.com> , "Gabriel Tanase"
            <gabtanase@...> wrote:
            >
            > Nice little article, however I can't resist a small complaint.
            > It doesn't make any reference to and makes no explicit provision for
            > multi-user access issues, which IMHO is a must in any non-trivial
            db-based
            > development.
            >
            > For example, the test
            shouldAssignPrimaryKeyValuesFromSequence()should be
            > designed to be an "atomic transaction", e.g. as below (apologies for the
            > bastardized syntax; I'm not a Java programmer):
            >
            > @Test
            > public void shouldAssignPrimaryKeyValuesFromSequence() {
            > *new lock_flag = Lock_Movie_PK_Context();*
            > Movie movie = createAndSave("PKASSIGNED");
            > Long currentPKValue = getCurrentValueForSequence("S_MOVIE");
            > *lock_flag = Unlock_Movie_PK_Context();*
            > assertNotNull("Movie should have been assigned a ID", movie.getId());
            > assertEquals("ID should have been same",movie.getId(),currentPKValue);
            >
            >
            > The problem is that in a multi-user environment the movie id value
            created
            > by
            > Movie movie = createAndSave("PKASSIGNED");
            > and stored into the movie object may be different from the one then
            > retrieved by
            > Long currentPKValue = getCurrentValueForSequence("S_MOVIE");
            > since another user may have created another movie, hence may have
            already
            > used a new sequence value, between the two statements.
            > Therefore the test would appear to fail (occasionally) purely
            because it is
            > executed in a multi-user environment.
            >
            >
            > Regards,
            > Gabriel
            >
            > 2009/1/5 ginitram <ginitram@...>
            >
            > > The Methods & Tools newsletter has released in its html archive
            section
            > > the article "Behavior Driven Database Development" by Pramod Sadalage.
            > >
            > > This article shows how the behavior driven development can be applied
            > > to database development and how it can be used to develop and design
            > > databases in an iterative and incremental way.
            > >
            > > http://www.methodsandtools.com/archive/archive.php?id=78
            > >
            > > .
            > >
            > >
            > >
            >
            >
            > [Non-text portions of this message have been removed]
            >





            [Non-text portions of this message have been removed]
          • Curt Sampson
            ... For each individual test to do this is nice in theory, but in practice it tends to be extremely slow. In QAM (http://www.starling-software.com/en/qam.html)
            Message 5 of 7 , Jan 6, 2009
            • 0 Attachment
              On 2009-01-06 10:04 -0800 (Tue), Max Guernsey, III wrote:

              > Should not the tests be responsible for creating their own database
              > instances?

              For each individual test to do this is nice in theory, but in practice
              it tends to be extremely slow.

              In QAM (http://www.starling-software.com/en/qam.html) we're
              developing a tool called qdb, which is a Ruby version of pgtools
              (http://www.starling-software.com/en/pgtools.html). (Pgtools runs only
              against PostgreSQL, qdb is much more primitive and runs only against
              MySQL but will be extended to PostgreSQL as we find the need.)

              Anyway, we have two standard schemas (in PostgreSQL) or databases (in
              MySQL).

              One is the functest_foo schema, which is wiped and reloaded afresh
              before a series of functional tests, which involve loading databases,
              starting web servers and daemons and so on, and doing things like
              sending a web request and checking both the page that comes back and the
              contents of the database afterwards.

              The other is the unittest_foo schema, which is reloaded only when
              the developer explicitly decides to do so. This is what we use for
              unit tests that need to touch a database (the tests for, e.g., stored
              procedures as well as a few tests of code accessing the database). The
              rule here is that you must leave the database exactly as you found it.
              Ideally, this is done by starting a transaction, making your changes,
              checking them, and then rolling back the transaction. However, sometimes
              you have to commit to make sure that something's working, in which case
              it's the responsibility of the developer of the test to make sure that
              it catches all errors and does what it needs to revert the database back
              to the pristine state.

              cjs
              --
              Curt Sampson <cjs@...> +81 90 7737 2974
              Functional programming in all senses of the word:
              http://www.starling-software.com
            • Max Guernsey, III
              For each individual test to do this is nice in theory, but in practice it tends to be extremely slow. That certainly does not corroborate with my experience.
              Message 6 of 7 , Jan 7, 2009
              • 0 Attachment
                "For each individual test to do this is nice in theory, but in practice
                it tends to be extremely slow."



                That certainly does not corroborate with my experience.



                What is "quite slow" in this case? .and, presuming you mean something more
                than a large fraction of a second per test, why is it so slow?



                Max Guernsey, III

                Managing Member, Hexagon Software

                <http://www.hexsw.com/> http://www.hexsw.com

                <http://www.dataconstructor.com/> http://www.dataconstructor.com



                From: agileDatabases@yahoogroups.com [mailto:agileDatabases@yahoogroups.com]
                On Behalf Of Curt Sampson
                Sent: Tuesday, January 06, 2009 9:02 PM
                To: agileDatabases@yahoogroups.com
                Subject: Re: [agileDatabases] Re: Behavior Driven Database Development



                On 2009-01-06 10:04 -0800 (Tue), Max Guernsey, III wrote:

                > Should not the tests be responsible for creating their own database
                > instances?

                For each individual test to do this is nice in theory, but in practice
                it tends to be extremely slow.

                In QAM (http://www.starling-software.com/en/qam.html) we're
                developing a tool called qdb, which is a Ruby version of pgtools
                (http://www.starling-software.com/en/pgtools.html). (Pgtools runs only
                against PostgreSQL, qdb is much more primitive and runs only against
                MySQL but will be extended to PostgreSQL as we find the need.)

                Anyway, we have two standard schemas (in PostgreSQL) or databases (in
                MySQL).

                One is the functest_foo schema, which is wiped and reloaded afresh
                before a series of functional tests, which involve loading databases,
                starting web servers and daemons and so on, and doing things like
                sending a web request and checking both the page that comes back and the
                contents of the database afterwards.

                The other is the unittest_foo schema, which is reloaded only when
                the developer explicitly decides to do so. This is what we use for
                unit tests that need to touch a database (the tests for, e.g., stored
                procedures as well as a few tests of code accessing the database). The
                rule here is that you must leave the database exactly as you found it.
                Ideally, this is done by starting a transaction, making your changes,
                checking them, and then rolling back the transaction. However, sometimes
                you have to commit to make sure that something's working, in which case
                it's the responsibility of the developer of the test to make sure that
                it catches all errors and does what it needs to revert the database back
                to the pristine state.

                cjs
                --
                Curt Sampson <cjs@... <mailto:cjs%40starling-software.com>
                > +81 90 7737 2974
                Functional programming in all senses of the word:
                http://www.starling-software.com





                [Non-text portions of this message have been removed]
              • Curt Sampson
                ... I mean a large fraction of a second. I don t do large schemas; I m generally working in the area of 20-200 tables. But even that, if you re going to run
                Message 7 of 7 , Jan 7, 2009
                • 0 Attachment
                  On 2009-01-07 10:05 -0800 (Wed), Max Guernsey, III wrote:

                  > "For each individual test to do this is nice in theory, but in practice
                  > it tends to be extremely slow."
                  > ...
                  > What is "quite slow" in this case? .and, presuming you mean something more
                  > than a large fraction of a second per test, why is it so slow?

                  I mean a large fraction of a second.

                  I don't do large schemas; I'm generally working in the area of 20-200
                  tables. But even that, if you're going to run eighty or a hundred unit
                  tests, the "recreate the entire databse" time is going to be noticable.
                  Even at a quarter second per database-create-load-test-teardown, that's
                  twenty seconds, which means it's not something you're going to run every
                  two minutes before every code commit.

                  If you can take sixty of those tests, and, with a little care, make them
                  run in two seconds rather than fifteen, why wouldn't you do that?

                  cjs
                  --
                  Curt Sampson <cjs@...> +81 90 7737 2974
                  Functional programming in all senses of the word:
                  http://www.starling-software.com
                Your message has been successfully submitted and would be delivered to recipients shortly.