Loading ...
Sorry, an error occurred while loading the content.

Tools to automatically generate "unit tests" source code

Expand Messages
  • M. Manca
    In the embedded tool market now there are companies promoting their (old) traditional products as useful to shorten the agile development times. Their theory
    Message 1 of 10 , Mar 20, 2012
    • 0 Attachment
      In the embedded tool market now there are companies promoting their
      (old) traditional products as useful to shorten the agile development
      times. Their theory (obviously wrong) is that adding more SLOC or
      function points to the software needed directly by the application
      causes a cost increasing proportional to the ratio of the SLOCs fir the
      (unit) tests and the application's SLOCs.

      So their are saying that auto generate unit test code (obviously you
      need existent code to test) guarantees a minor cost and the same quality
      of the agile unit test without their higher costs.

      I am sure that this is totally wrong my question is: anyone knows some
      articles or materials about this argument (because I am sure that this
      is not the 1st time someone said this) and articles to contest this poor
      vision about unit tests (more formally correct: developer tests in this
      situation)?



      [Non-text portions of this message have been removed]
    • George Dinwiddie
      ... I wouldn t know where to start on writing such an article. How can an automated tool know what the programmer intended? I looked at one such tool, years
      Message 2 of 10 , Mar 20, 2012
      • 0 Attachment
        On 3/20/12 12:32 PM, M. Manca wrote:
        > In the embedded tool market now there are companies promoting their
        > (old) traditional products as useful to shorten the agile development
        > times. Their theory (obviously wrong) is that adding more SLOC or
        > function points to the software needed directly by the application
        > causes a cost increasing proportional to the ratio of the SLOCs fir the
        > (unit) tests and the application's SLOCs.
        >
        > So their are saying that auto generate unit test code (obviously you
        > need existent code to test) guarantees a minor cost and the same quality
        > of the agile unit test without their higher costs.
        >
        > I am sure that this is totally wrong my question is: anyone knows some
        > articles or materials about this argument (because I am sure that this
        > is not the 1st time someone said this) and articles to contest this poor
        > vision about unit tests (more formally correct: developer tests in this
        > situation)?

        I wouldn't know where to start on writing such an article. How can an
        automated tool know what the programmer intended?

        I looked at one such tool, years back, and all it did was pass nulls as
        each argument to a method. Since my code used the Null Object Pattern,
        this was exactly contrary to my intent.

        - George

        --
        ----------------------------------------------------------------------
        * George Dinwiddie * http://blog.gdinwiddie.com
        Software Development http://www.idiacomputing.com
        Consultant and Coach http://www.agilemaryland.org
        ----------------------------------------------------------------------
      • Charlie Poole
        My feeling is that test-generation *might* be useful in *some* applications after the code was developed using TDD. Some apps (fewer than many people think)
        Message 3 of 10 , Mar 20, 2012
        • 0 Attachment
          My feeling is that test-generation *might* be useful in *some* applications
          after the code was developed using TDD.

          Some apps (fewer than many people think) need more testing after the code
          is developed using TDD. It needs a very sophisticated implementation to do
          this effectively. One such implemenation is the human brain. Conceivably,
          an app could use code analysis and numerical analysis to find test cases
          that would add value. As an alternative, one could use brute force to
          generate large numbers of such cases, but this isn't so satisfactory.

          My guess is that such an approach would be useful - if it existed - in less
          than 1% of the projects I've seen. A naive approach to test generation
          won't be useful anywhere but has the danger that it might pull people away
          from doing what they need to be doing: writing the tests themselves.

          If I had a choice between two magical apps, one generating tests from
          source code and one generating the apps from the tests, I'd pick the second
          and write the tests. AFAIK, neither is on offer. :-(

          Charlie

          On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
          <lists@...>wrote:

          > **
          >
          >
          > On 3/20/12 12:32 PM, M. Manca wrote:
          > > In the embedded tool market now there are companies promoting their
          > > (old) traditional products as useful to shorten the agile development
          > > times. Their theory (obviously wrong) is that adding more SLOC or
          > > function points to the software needed directly by the application
          > > causes a cost increasing proportional to the ratio of the SLOCs fir the
          > > (unit) tests and the application's SLOCs.
          > >
          > > So their are saying that auto generate unit test code (obviously you
          > > need existent code to test) guarantees a minor cost and the same quality
          > > of the agile unit test without their higher costs.
          > >
          > > I am sure that this is totally wrong my question is: anyone knows some
          > > articles or materials about this argument (because I am sure that this
          > > is not the 1st time someone said this) and articles to contest this poor
          > > vision about unit tests (more formally correct: developer tests in this
          > > situation)?
          >
          > I wouldn't know where to start on writing such an article. How can an
          > automated tool know what the programmer intended?
          >
          > I looked at one such tool, years back, and all it did was pass nulls as
          > each argument to a method. Since my code used the Null Object Pattern,
          > this was exactly contrary to my intent.
          >
          > - George
          >
          > --
          > ----------------------------------------------------------
          > * George Dinwiddie * http://blog.gdinwiddie.com
          > Software Development http://www.idiacomputing.com
          > Consultant and Coach http://www.agilemaryland.org
          > ----------------------------------------------------------
          >
          >
          >


          [Non-text portions of this message have been removed]
        • Daniel Wildt
          I know this one... http://www.agitar.com/solutions/products/automated_junit_generation.html -- Daniel Wildt (+55 51 99989030) - @dwildt
          Message 4 of 10 , Mar 20, 2012
          • 0 Attachment
            I know this one...
            http://www.agitar.com/solutions/products/automated_junit_generation.html


            -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>


            On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...>wrote:

            > My feeling is that test-generation *might* be useful in *some* applications
            > after the code was developed using TDD.
            >
            > Some apps (fewer than many people think) need more testing after the code
            > is developed using TDD. It needs a very sophisticated implementation to do
            > this effectively. One such implemenation is the human brain. Conceivably,
            > an app could use code analysis and numerical analysis to find test cases
            > that would add value. As an alternative, one could use brute force to
            > generate large numbers of such cases, but this isn't so satisfactory.
            >
            > My guess is that such an approach would be useful - if it existed - in less
            > than 1% of the projects I've seen. A naive approach to test generation
            > won't be useful anywhere but has the danger that it might pull people away
            > from doing what they need to be doing: writing the tests themselves.
            >
            > If I had a choice between two magical apps, one generating tests from
            > source code and one generating the apps from the tests, I'd pick the second
            > and write the tests. AFAIK, neither is on offer. :-(
            >
            > Charlie
            >
            > On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
            > <lists@...>wrote:
            >
            > > **
            > >
            > >
            > > On 3/20/12 12:32 PM, M. Manca wrote:
            > > > In the embedded tool market now there are companies promoting their
            > > > (old) traditional products as useful to shorten the agile development
            > > > times. Their theory (obviously wrong) is that adding more SLOC or
            > > > function points to the software needed directly by the application
            > > > causes a cost increasing proportional to the ratio of the SLOCs fir the
            > > > (unit) tests and the application's SLOCs.
            > > >
            > > > So their are saying that auto generate unit test code (obviously you
            > > > need existent code to test) guarantees a minor cost and the same
            > quality
            > > > of the agile unit test without their higher costs.
            > > >
            > > > I am sure that this is totally wrong my question is: anyone knows some
            > > > articles or materials about this argument (because I am sure that this
            > > > is not the 1st time someone said this) and articles to contest this
            > poor
            > > > vision about unit tests (more formally correct: developer tests in this
            > > > situation)?
            > >
            > > I wouldn't know where to start on writing such an article. How can an
            > > automated tool know what the programmer intended?
            > >
            > > I looked at one such tool, years back, and all it did was pass nulls as
            > > each argument to a method. Since my code used the Null Object Pattern,
            > > this was exactly contrary to my intent.
            > >
            > > - George
            > >
            > > --
            > > ----------------------------------------------------------
            > > * George Dinwiddie * http://blog.gdinwiddie.com
            > > Software Development http://www.idiacomputing.com
            > > Consultant and Coach http://www.agilemaryland.org
            > > ----------------------------------------------------------
            > >
            > >
            > >
            >
            >
            > [Non-text portions of this message have been removed]
            >
            >
            >
            > ------------------------------------
            >
            > To Post a message, send it to: extremeprogramming@...
            >
            > To Unsubscribe, send a blank message to:
            > extremeprogramming-unsubscribe@...
            >
            > ad-free courtesy of objectmentor.comYahoo! Groups Links
            >
            >
            >
            >


            [Non-text portions of this message have been removed]
          • M. Manca
            Il 20/03/2012 18:41, Daniel Wildt ha scritto: I need to correct myself: when I wrote article I was meaning a magazine article not a software tool or product.
            Message 5 of 10 , Mar 20, 2012
            • 0 Attachment
              Il 20/03/2012 18:41, Daniel Wildt ha scritto:
              I need to correct myself: when I wrote "article" I was meaning a
              magazine article not a software tool or product.

              So I would find to try some documentation favorable and contrary to the
              automatic unit test generation.

              My opinion is that a unit test has to be written before the code to test
              in a TDD or BDD fashion.
              The advantage is embedded in the order of things, writing unit tests
              after the source code is just a new form of developer tests, writing the
              test before is a way to transform a written requirement in an executable
              requirement and this is the very big advantage.

              I feel very natural to write the test before the code that implements
              the behavior.
              >
              >
              > I know this one...
              > http://www.agitar.com/solutions/products/automated_junit_generation.html
              >
              > -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>
              >
              > On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...
              > <mailto:charliepoole%40gmail.com>>wrote:
              >
              > > My feeling is that test-generation *might* be useful in *some*
              > applications
              > > after the code was developed using TDD.
              > >
              > > Some apps (fewer than many people think) need more testing after the
              > code
              > > is developed using TDD. It needs a very sophisticated implementation
              > to do
              > > this effectively. One such implemenation is the human brain.
              > Conceivably,
              > > an app could use code analysis and numerical analysis to find test cases
              > > that would add value. As an alternative, one could use brute force to
              > > generate large numbers of such cases, but this isn't so satisfactory.
              > >
              > > My guess is that such an approach would be useful - if it existed -
              > in less
              > > than 1% of the projects I've seen. A naive approach to test generation
              > > won't be useful anywhere but has the danger that it might pull
              > people away
              > > from doing what they need to be doing: writing the tests themselves.
              > >
              > > If I had a choice between two magical apps, one generating tests from
              > > source code and one generating the apps from the tests, I'd pick the
              > second
              > > and write the tests. AFAIK, neither is on offer. :-(
              > >
              > > Charlie
              > >
              > > On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
              > > <lists@... <mailto:lists%40idiacomputing.com>>wrote:
              > >
              > > > **
              > > >
              > > >
              > > > On 3/20/12 12:32 PM, M. Manca wrote:
              > > > > In the embedded tool market now there are companies promoting their
              > > > > (old) traditional products as useful to shorten the agile
              > development
              > > > > times. Their theory (obviously wrong) is that adding more SLOC or
              > > > > function points to the software needed directly by the application
              > > > > causes a cost increasing proportional to the ratio of the SLOCs
              > fir the
              > > > > (unit) tests and the application's SLOCs.
              > > > >
              > > > > So their are saying that auto generate unit test code (obviously you
              > > > > need existent code to test) guarantees a minor cost and the same
              > > quality
              > > > > of the agile unit test without their higher costs.
              > > > >
              > > > > I am sure that this is totally wrong my question is: anyone
              > knows some
              > > > > articles or materials about this argument (because I am sure
              > that this
              > > > > is not the 1st time someone said this) and articles to contest this
              > > poor
              > > > > vision about unit tests (more formally correct: developer tests
              > in this
              > > > > situation)?
              > > >
              > > > I wouldn't know where to start on writing such an article. How can an
              > > > automated tool know what the programmer intended?
              > > >
              > > > I looked at one such tool, years back, and all it did was pass
              > nulls as
              > > > each argument to a method. Since my code used the Null Object Pattern,
              > > > this was exactly contrary to my intent.
              > > >
              > > > - George
              > > >
              > > > --
              > > > ----------------------------------------------------------
              > > > * George Dinwiddie * http://blog.gdinwiddie.com
              > > > Software Development http://www.idiacomputing.com
              > > > Consultant and Coach http://www.agilemaryland.org
              > > > ----------------------------------------------------------
              > > >
              > > >
              > > >
              > >
              > >
              > > [Non-text portions of this message have been removed]
              > >
              > >
              > >
              > > ------------------------------------
              > >
              > > To Post a message, send it to: extremeprogramming@...
              > <mailto:extremeprogramming%40eGroups.com>
              > >
              > > To Unsubscribe, send a blank message to:
              > > extremeprogramming-unsubscribe@...
              > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
              > >
              > > ad-free courtesy of objectmentor.comYahoo! Groups Links
              > >
              > >
              > >
              > >
              >
              > [Non-text portions of this message have been removed]
              >
              >



              [Non-text portions of this message have been removed]
            • Adam Sroka
              I have heard the argument before and I have seen some of the tools that attempt to do this. I haven t delved deeply into it, but my initial impression is that
              Message 6 of 10 , Mar 20, 2012
              • 0 Attachment
                I have heard the argument before and I have seen some of the tools that
                attempt to do this. I haven't delved deeply into it, but my initial
                impression is that these tools tend to create a bunch of brittle,
                uncommunicative tests. The tools lack the domain knowledge and intelligence
                necessary to craft well-written tests with meaningful assertions that serve
                as documentation of the code. If the only quality you care about in testing
                is coverage then you might find them useful.

                On Tue, Mar 20, 2012 at 11:36 AM, M. Manca <m.manca@...>wrote:

                > **
                >
                >
                > Il 20/03/2012 18:41, Daniel Wildt ha scritto:
                > I need to correct myself: when I wrote "article" I was meaning a
                > magazine article not a software tool or product.
                >
                > So I would find to try some documentation favorable and contrary to the
                > automatic unit test generation.
                >
                > My opinion is that a unit test has to be written before the code to test
                > in a TDD or BDD fashion.
                > The advantage is embedded in the order of things, writing unit tests
                > after the source code is just a new form of developer tests, writing the
                > test before is a way to transform a written requirement in an executable
                > requirement and this is the very big advantage.
                >
                > I feel very natural to write the test before the code that implements
                > the behavior.
                >
                > >
                > >
                > > I know this one...
                > > http://www.agitar.com/solutions/products/automated_junit_generation.html
                > >
                > > -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>
                > >
                > > On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...
                > > <mailto:charliepoole%40gmail.com>>wrote:
                >
                > >
                > > > My feeling is that test-generation *might* be useful in *some*
                > > applications
                > > > after the code was developed using TDD.
                > > >
                > > > Some apps (fewer than many people think) need more testing after the
                > > code
                > > > is developed using TDD. It needs a very sophisticated implementation
                > > to do
                > > > this effectively. One such implemenation is the human brain.
                > > Conceivably,
                > > > an app could use code analysis and numerical analysis to find test
                > cases
                > > > that would add value. As an alternative, one could use brute force to
                > > > generate large numbers of such cases, but this isn't so satisfactory.
                > > >
                > > > My guess is that such an approach would be useful - if it existed -
                > > in less
                > > > than 1% of the projects I've seen. A naive approach to test generation
                > > > won't be useful anywhere but has the danger that it might pull
                > > people away
                > > > from doing what they need to be doing: writing the tests themselves.
                > > >
                > > > If I had a choice between two magical apps, one generating tests from
                > > > source code and one generating the apps from the tests, I'd pick the
                > > second
                > > > and write the tests. AFAIK, neither is on offer. :-(
                > > >
                > > > Charlie
                > > >
                > > > On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
                > > > <lists@... <mailto:lists%40idiacomputing.com>>wrote:
                >
                > > >
                > > > > **
                > > > >
                > > > >
                > > > > On 3/20/12 12:32 PM, M. Manca wrote:
                > > > > > In the embedded tool market now there are companies promoting their
                > > > > > (old) traditional products as useful to shorten the agile
                > > development
                > > > > > times. Their theory (obviously wrong) is that adding more SLOC or
                > > > > > function points to the software needed directly by the application
                > > > > > causes a cost increasing proportional to the ratio of the SLOCs
                > > fir the
                > > > > > (unit) tests and the application's SLOCs.
                > > > > >
                > > > > > So their are saying that auto generate unit test code (obviously
                > you
                > > > > > need existent code to test) guarantees a minor cost and the same
                > > > quality
                > > > > > of the agile unit test without their higher costs.
                > > > > >
                > > > > > I am sure that this is totally wrong my question is: anyone
                > > knows some
                > > > > > articles or materials about this argument (because I am sure
                > > that this
                > > > > > is not the 1st time someone said this) and articles to contest this
                > > > poor
                > > > > > vision about unit tests (more formally correct: developer tests
                > > in this
                > > > > > situation)?
                > > > >
                > > > > I wouldn't know where to start on writing such an article. How can an
                > > > > automated tool know what the programmer intended?
                > > > >
                > > > > I looked at one such tool, years back, and all it did was pass
                > > nulls as
                > > > > each argument to a method. Since my code used the Null Object
                > Pattern,
                > > > > this was exactly contrary to my intent.
                > > > >
                > > > > - George
                > > > >
                > > > > --
                > > > > ----------------------------------------------------------
                > > > > * George Dinwiddie * http://blog.gdinwiddie.com
                > > > > Software Development http://www.idiacomputing.com
                > > > > Consultant and Coach http://www.agilemaryland.org
                > > > > ----------------------------------------------------------
                > > > >
                > > > >
                > > > >
                > > >
                > > >
                > > > [Non-text portions of this message have been removed]
                > > >
                > > >
                > > >
                > > > ------------------------------------
                > > >
                > > > To Post a message, send it to: extremeprogramming@...
                > > <mailto:extremeprogramming%40eGroups.com>
                >
                > > >
                > > > To Unsubscribe, send a blank message to:
                > > > extremeprogramming-unsubscribe@...
                > > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                >
                > > >
                > > > ad-free courtesy of objectmentor.comYahoo! Groups Links
                > > >
                > > >
                > > >
                > > >
                > >
                > > [Non-text portions of this message have been removed]
                > >
                > >
                >
                > [Non-text portions of this message have been removed]
                >
                >
                >


                [Non-text portions of this message have been removed]
              • james@renaissancesoftware.net
                Funny, I had a client using tools like this last week. I agree with Adam that these tests are of little value. Aside from helping achieve a metric (that
                Message 7 of 10 , Mar 20, 2012
                • 0 Attachment
                  Funny, I had a client using tools like this last week. I agree with Adam that these tests are of little value. Aside from helping achieve a metric (that should not be the goal anyway), these generated tests might be helpful when making sure you understand the consequences when changing legacy code. They cannot provide the other benefits from hand crafted tests and TDD.

                  thanks, James

                  --------------------------------------------------------------------------------------------
                  James Grenning Author of TDD for Embedded C
                  www.renaissancesoftware.net http://pragprog.com/titles/jgade/
                  www.renaissancesoftware.net/blog
                  www.twitter.com/jwgrenning


                  On Mar 20, 2012, at 1:54 PM, Adam Sroka wrote:

                  > I have heard the argument before and I have seen some of the tools that
                  > attempt to do this. I haven't delved deeply into it, but my initial
                  > impression is that these tools tend to create a bunch of brittle,
                  > uncommunicative tests. The tools lack the domain knowledge and intelligence
                  > necessary to craft well-written tests with meaningful assertions that serve
                  > as documentation of the code. If the only quality you care about in testing
                  > is coverage then you might find them useful.
                  >
                  > On Tue, Mar 20, 2012 at 11:36 AM, M. Manca <m.manca@...>wrote:
                  >
                  >> **
                  >>
                  >>
                  >> Il 20/03/2012 18:41, Daniel Wildt ha scritto:
                  >> I need to correct myself: when I wrote "article" I was meaning a
                  >> magazine article not a software tool or product.
                  >>
                  >> So I would find to try some documentation favorable and contrary to the
                  >> automatic unit test generation.
                  >>
                  >> My opinion is that a unit test has to be written before the code to test
                  >> in a TDD or BDD fashion.
                  >> The advantage is embedded in the order of things, writing unit tests
                  >> after the source code is just a new form of developer tests, writing the
                  >> test before is a way to transform a written requirement in an executable
                  >> requirement and this is the very big advantage.
                  >>
                  >> I feel very natural to write the test before the code that implements
                  >> the behavior.
                  >>
                  >>>
                  >>>
                  >>> I know this one...
                  >>> http://www.agitar.com/solutions/products/automated_junit_generation.html
                  >>>
                  >>> -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>
                  >>>
                  >>> On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...
                  >>> <mailto:charliepoole%40gmail.com>>wrote:
                  >>
                  >>>
                  >>>> My feeling is that test-generation *might* be useful in *some*
                  >>> applications
                  >>>> after the code was developed using TDD.
                  >>>>
                  >>>> Some apps (fewer than many people think) need more testing after the
                  >>> code
                  >>>> is developed using TDD. It needs a very sophisticated implementation
                  >>> to do
                  >>>> this effectively. One such implemenation is the human brain.
                  >>> Conceivably,
                  >>>> an app could use code analysis and numerical analysis to find test
                  >> cases
                  >>>> that would add value. As an alternative, one could use brute force to
                  >>>> generate large numbers of such cases, but this isn't so satisfactory.
                  >>>>
                  >>>> My guess is that such an approach would be useful - if it existed -
                  >>> in less
                  >>>> than 1% of the projects I've seen. A naive approach to test generation
                  >>>> won't be useful anywhere but has the danger that it might pull
                  >>> people away
                  >>>> from doing what they need to be doing: writing the tests themselves.
                  >>>>
                  >>>> If I had a choice between two magical apps, one generating tests from
                  >>>> source code and one generating the apps from the tests, I'd pick the
                  >>> second
                  >>>> and write the tests. AFAIK, neither is on offer. :-(
                  >>>>
                  >>>> Charlie
                  >>>>
                  >>>> On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
                  >>>> <lists@... <mailto:lists%40idiacomputing.com>>wrote:
                  >>
                  >>>>
                  >>>>> **
                  >>>>>
                  >>>>>
                  >>>>> On 3/20/12 12:32 PM, M. Manca wrote:
                  >>>>>> In the embedded tool market now there are companies promoting their
                  >>>>>> (old) traditional products as useful to shorten the agile
                  >>> development
                  >>>>>> times. Their theory (obviously wrong) is that adding more SLOC or
                  >>>>>> function points to the software needed directly by the application
                  >>>>>> causes a cost increasing proportional to the ratio of the SLOCs
                  >>> fir the
                  >>>>>> (unit) tests and the application's SLOCs.
                  >>>>>>
                  >>>>>> So their are saying that auto generate unit test code (obviously
                  >> you
                  >>>>>> need existent code to test) guarantees a minor cost and the same
                  >>>> quality
                  >>>>>> of the agile unit test without their higher costs.
                  >>>>>>
                  >>>>>> I am sure that this is totally wrong my question is: anyone
                  >>> knows some
                  >>>>>> articles or materials about this argument (because I am sure
                  >>> that this
                  >>>>>> is not the 1st time someone said this) and articles to contest this
                  >>>> poor
                  >>>>>> vision about unit tests (more formally correct: developer tests
                  >>> in this
                  >>>>>> situation)?
                  >>>>>
                  >>>>> I wouldn't know where to start on writing such an article. How can an
                  >>>>> automated tool know what the programmer intended?
                  >>>>>
                  >>>>> I looked at one such tool, years back, and all it did was pass
                  >>> nulls as
                  >>>>> each argument to a method. Since my code used the Null Object
                  >> Pattern,
                  >>>>> this was exactly contrary to my intent.
                  >>>>>
                  >>>>> - George
                  >>>>>
                  >>>>> --
                  >>>>> ----------------------------------------------------------
                  >>>>> * George Dinwiddie * http://blog.gdinwiddie.com
                  >>>>> Software Development http://www.idiacomputing.com
                  >>>>> Consultant and Coach http://www.agilemaryland.org
                  >>>>> ----------------------------------------------------------
                  >>>>>
                  >>>>>
                  >>>>>
                  >>>>
                  >>>>
                  >>>> [Non-text portions of this message have been removed]
                  >>>>
                  >>>>
                  >>>>
                  >>>> ------------------------------------
                  >>>>
                  >>>> To Post a message, send it to: extremeprogramming@...
                  >>> <mailto:extremeprogramming%40eGroups.com>
                  >>
                  >>>>
                  >>>> To Unsubscribe, send a blank message to:
                  >>>> extremeprogramming-unsubscribe@...
                  >>> <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                  >>
                  >>>>
                  >>>> ad-free courtesy of objectmentor.comYahoo! Groups Links
                  >>>>
                  >>>>
                  >>>>
                  >>>>
                  >>>
                  >>> [Non-text portions of this message have been removed]
                  >>>
                  >>>
                  >>
                  >> [Non-text portions of this message have been removed]
                  >>
                  >>
                  >>
                  >
                  >
                  > [Non-text portions of this message have been removed]
                  >
                  >
                  >
                  > ------------------------------------
                  >
                  > To Post a message, send it to: extremeprogramming@...
                  >
                  > To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
                  >
                  > ad-free courtesy of objectmentor.comYahoo! Groups Links
                  >
                  >
                  >
                • M. Manca
                  ... So we all agree that these tools have little value. This is a good thing, now how can we communicate this idea to the embedded developers community?
                  Message 8 of 10 , Mar 20, 2012
                  • 0 Attachment
                    Il 20/03/2012 20:28, james@... ha scritto:
                    >
                    >
                    > Funny, I had a client using tools like this last week. I agree with
                    > Adam that these tests are of little value. Aside from helping achieve
                    > a metric (that should not be the goal anyway), these generated tests
                    > might be helpful when making sure you understand the consequences when
                    > changing legacy code. They cannot provide the other benefits from hand
                    > crafted tests and TDD.
                    >
                    > thanks, James
                    >
                    So we all agree that these tools have little value.
                    This is a good thing, now how can we communicate this idea to the
                    embedded developers community? Because if no one say that these tools
                    have no value the idea that will pass is that they help to deliver
                    better software.
                    >
                    >
                    > ----------------------------------------------------------
                    > James Grenning Author of TDD for Embedded C
                    > www.renaissancesoftware.net http://pragprog.com/titles/jgade/
                    > www.renaissancesoftware.net/blog
                    > www.twitter.com/jwgrenning
                    >
                    > On Mar 20, 2012, at 1:54 PM, Adam Sroka wrote:
                    >
                    > > I have heard the argument before and I have seen some of the tools that
                    > > attempt to do this. I haven't delved deeply into it, but my initial
                    > > impression is that these tools tend to create a bunch of brittle,
                    > > uncommunicative tests. The tools lack the domain knowledge and
                    > intelligence
                    > > necessary to craft well-written tests with meaningful assertions
                    > that serve
                    > > as documentation of the code. If the only quality you care about in
                    > testing
                    > > is coverage then you might find them useful.
                    > >
                    > > On Tue, Mar 20, 2012 at 11:36 AM, M. Manca
                    > <m.manca@...
                    > <mailto:m.manca%40micronengineering.it>>wrote:
                    > >
                    > >> **
                    > >>
                    > >>
                    > >> Il 20/03/2012 18:41, Daniel Wildt ha scritto:
                    > >> I need to correct myself: when I wrote "article" I was meaning a
                    > >> magazine article not a software tool or product.
                    > >>
                    > >> So I would find to try some documentation favorable and contrary to the
                    > >> automatic unit test generation.
                    > >>
                    > >> My opinion is that a unit test has to be written before the code to
                    > test
                    > >> in a TDD or BDD fashion.
                    > >> The advantage is embedded in the order of things, writing unit tests
                    > >> after the source code is just a new form of developer tests,
                    > writing the
                    > >> test before is a way to transform a written requirement in an
                    > executable
                    > >> requirement and this is the very big advantage.
                    > >>
                    > >> I feel very natural to write the test before the code that implements
                    > >> the behavior.
                    > >>
                    > >>>
                    > >>>
                    > >>> I know this one...
                    > >>>
                    > http://www.agitar.com/solutions/products/automated_junit_generation.html
                    > >>>
                    > >>> -- Daniel Wildt (+55 51 99989030) - @dwildt
                    > <http://twitter.com/dwildt>
                    > >>>
                    > >>> On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole
                    > <charliepoole@... <mailto:charliepoole%40gmail.com>
                    > >>> <mailto:charliepoole%40gmail.com>>wrote:
                    > >>
                    > >>>
                    > >>>> My feeling is that test-generation *might* be useful in *some*
                    > >>> applications
                    > >>>> after the code was developed using TDD.
                    > >>>>
                    > >>>> Some apps (fewer than many people think) need more testing after the
                    > >>> code
                    > >>>> is developed using TDD. It needs a very sophisticated implementation
                    > >>> to do
                    > >>>> this effectively. One such implemenation is the human brain.
                    > >>> Conceivably,
                    > >>>> an app could use code analysis and numerical analysis to find test
                    > >> cases
                    > >>>> that would add value. As an alternative, one could use brute force to
                    > >>>> generate large numbers of such cases, but this isn't so satisfactory.
                    > >>>>
                    > >>>> My guess is that such an approach would be useful - if it existed -
                    > >>> in less
                    > >>>> than 1% of the projects I've seen. A naive approach to test
                    > generation
                    > >>>> won't be useful anywhere but has the danger that it might pull
                    > >>> people away
                    > >>>> from doing what they need to be doing: writing the tests themselves.
                    > >>>>
                    > >>>> If I had a choice between two magical apps, one generating tests from
                    > >>>> source code and one generating the apps from the tests, I'd pick the
                    > >>> second
                    > >>>> and write the tests. AFAIK, neither is on offer. :-(
                    > >>>>
                    > >>>> Charlie
                    > >>>>
                    > >>>> On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
                    > >>>> <lists@... <mailto:lists%40idiacomputing.com>
                    > <mailto:lists%40idiacomputing.com>>wrote:
                    > >>
                    > >>>>
                    > >>>>> **
                    > >>>>>
                    > >>>>>
                    > >>>>> On 3/20/12 12:32 PM, M. Manca wrote:
                    > >>>>>> In the embedded tool market now there are companies promoting their
                    > >>>>>> (old) traditional products as useful to shorten the agile
                    > >>> development
                    > >>>>>> times. Their theory (obviously wrong) is that adding more SLOC or
                    > >>>>>> function points to the software needed directly by the application
                    > >>>>>> causes a cost increasing proportional to the ratio of the SLOCs
                    > >>> fir the
                    > >>>>>> (unit) tests and the application's SLOCs.
                    > >>>>>>
                    > >>>>>> So their are saying that auto generate unit test code (obviously
                    > >> you
                    > >>>>>> need existent code to test) guarantees a minor cost and the same
                    > >>>> quality
                    > >>>>>> of the agile unit test without their higher costs.
                    > >>>>>>
                    > >>>>>> I am sure that this is totally wrong my question is: anyone
                    > >>> knows some
                    > >>>>>> articles or materials about this argument (because I am sure
                    > >>> that this
                    > >>>>>> is not the 1st time someone said this) and articles to contest this
                    > >>>> poor
                    > >>>>>> vision about unit tests (more formally correct: developer tests
                    > >>> in this
                    > >>>>>> situation)?
                    > >>>>>
                    > >>>>> I wouldn't know where to start on writing such an article. How
                    > can an
                    > >>>>> automated tool know what the programmer intended?
                    > >>>>>
                    > >>>>> I looked at one such tool, years back, and all it did was pass
                    > >>> nulls as
                    > >>>>> each argument to a method. Since my code used the Null Object
                    > >> Pattern,
                    > >>>>> this was exactly contrary to my intent.
                    > >>>>>
                    > >>>>> - George
                    > >>>>>
                    > >>>>> --
                    > >>>>> ----------------------------------------------------------
                    > >>>>> * George Dinwiddie * http://blog.gdinwiddie.com
                    > >>>>> Software Development http://www.idiacomputing.com
                    > >>>>> Consultant and Coach http://www.agilemaryland.org
                    > >>>>> ----------------------------------------------------------
                    > >>>>>
                    > >>>>>
                    > >>>>>
                    > >>>>
                    > >>>>
                    > >>>> [Non-text portions of this message have been removed]
                    > >>>>
                    > >>>>
                    > >>>>
                    > >>>> ------------------------------------
                    > >>>>
                    > >>>> To Post a message, send it to: extremeprogramming@...
                    > <mailto:extremeprogramming%40eGroups.com>
                    > >>> <mailto:extremeprogramming%40eGroups.com>
                    > >>
                    > >>>>
                    > >>>> To Unsubscribe, send a blank message to:
                    > >>>> extremeprogramming-unsubscribe@...
                    > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                    > >>> <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                    > >>
                    > >>>>
                    > >>>> ad-free courtesy of objectmentor.comYahoo! Groups Links
                    > >>>>
                    > >>>>
                    > >>>>
                    > >>>>
                    > >>>
                    > >>> [Non-text portions of this message have been removed]
                    > >>>
                    > >>>
                    > >>
                    > >> [Non-text portions of this message have been removed]
                    > >>
                    > >>
                    > >>
                    > >
                    > >
                    > > [Non-text portions of this message have been removed]
                    > >
                    > >
                    > >
                    > > ------------------------------------
                    > >
                    > > To Post a message, send it to: extremeprogramming@...
                    > <mailto:extremeprogramming%40eGroups.com>
                    > >
                    > > To Unsubscribe, send a blank message to:
                    > extremeprogramming-unsubscribe@...
                    > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                    > >
                    > > ad-free courtesy of objectmentor.comYahoo! Groups Links
                    > >
                    > >
                    > >
                    >
                    >



                    [Non-text portions of this message have been removed]
                  • james@renaissancesoftware.net
                    I might break the news to some of them at my Embedded Systems Conference sessions next week. ... [Non-text portions of this message have been removed]
                    Message 9 of 10 , Mar 20, 2012
                    • 0 Attachment
                      I might break the news to some of them at my Embedded Systems Conference sessions next week.


                      On Mar 20, 2012, at 2:42 PM, M. Manca wrote:

                      > Il 20/03/2012 20:28, james@... ha scritto:
                      > >
                      > >
                      > > Funny, I had a client using tools like this last week. I agree with
                      > > Adam that these tests are of little value. Aside from helping achieve
                      > > a metric (that should not be the goal anyway), these generated tests
                      > > might be helpful when making sure you understand the consequences when
                      > > changing legacy code. They cannot provide the other benefits from hand
                      > > crafted tests and TDD.
                      > >
                      > > thanks, James
                      > >
                      > So we all agree that these tools have little value.
                      > This is a good thing, now how can we communicate this idea to the
                      > embedded developers community? Because if no one say that these tools
                      > have no value the idea that will pass is that they help to deliver
                      > better software.
                      > >
                      > >
                      > > ----------------------------------------------------------
                      > > James Grenning Author of TDD for Embedded C
                      > > www.renaissancesoftware.net http://pragprog.com/titles/jgade/
                      > > www.renaissancesoftware.net/blog
                      > > www.twitter.com/jwgrenning
                      > >
                      > > On Mar 20, 2012, at 1:54 PM, Adam Sroka wrote:
                      > >
                      > > > I have heard the argument before and I have seen some of the tools that
                      > > > attempt to do this. I haven't delved deeply into it, but my initial
                      > > > impression is that these tools tend to create a bunch of brittle,
                      > > > uncommunicative tests. The tools lack the domain knowledge and
                      > > intelligence
                      > > > necessary to craft well-written tests with meaningful assertions
                      > > that serve
                      > > > as documentation of the code. If the only quality you care about in
                      > > testing
                      > > > is coverage then you might find them useful.
                      > > >
                      > > > On Tue, Mar 20, 2012 at 11:36 AM, M. Manca
                      > > <m.manca@...
                      > > <mailto:m.manca%40micronengineering.it>>wrote:
                      > > >
                      > > >> **
                      > > >>
                      > > >>
                      > > >> Il 20/03/2012 18:41, Daniel Wildt ha scritto:
                      > > >> I need to correct myself: when I wrote "article" I was meaning a
                      > > >> magazine article not a software tool or product.
                      > > >>
                      > > >> So I would find to try some documentation favorable and contrary to the
                      > > >> automatic unit test generation.
                      > > >>
                      > > >> My opinion is that a unit test has to be written before the code to
                      > > test
                      > > >> in a TDD or BDD fashion.
                      > > >> The advantage is embedded in the order of things, writing unit tests
                      > > >> after the source code is just a new form of developer tests,
                      > > writing the
                      > > >> test before is a way to transform a written requirement in an
                      > > executable
                      > > >> requirement and this is the very big advantage.
                      > > >>
                      > > >> I feel very natural to write the test before the code that implements
                      > > >> the behavior.
                      > > >>
                      > > >>>
                      > > >>>
                      > > >>> I know this one...
                      > > >>>
                      > > http://www.agitar.com/solutions/products/automated_junit_generation.html
                      > > >>>
                      > > >>> -- Daniel Wildt (+55 51 99989030) - @dwildt
                      > > <http://twitter.com/dwildt>
                      > > >>>
                      > > >>> On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole
                      > > <charliepoole@... <mailto:charliepoole%40gmail.com>
                      > > >>> <mailto:charliepoole%40gmail.com>>wrote:
                      > > >>
                      > > >>>
                      > > >>>> My feeling is that test-generation *might* be useful in *some*
                      > > >>> applications
                      > > >>>> after the code was developed using TDD.
                      > > >>>>
                      > > >>>> Some apps (fewer than many people think) need more testing after the
                      > > >>> code
                      > > >>>> is developed using TDD. It needs a very sophisticated implementation
                      > > >>> to do
                      > > >>>> this effectively. One such implemenation is the human brain.
                      > > >>> Conceivably,
                      > > >>>> an app could use code analysis and numerical analysis to find test
                      > > >> cases
                      > > >>>> that would add value. As an alternative, one could use brute force to
                      > > >>>> generate large numbers of such cases, but this isn't so satisfactory.
                      > > >>>>
                      > > >>>> My guess is that such an approach would be useful - if it existed -
                      > > >>> in less
                      > > >>>> than 1% of the projects I've seen. A naive approach to test
                      > > generation
                      > > >>>> won't be useful anywhere but has the danger that it might pull
                      > > >>> people away
                      > > >>>> from doing what they need to be doing: writing the tests themselves.
                      > > >>>>
                      > > >>>> If I had a choice between two magical apps, one generating tests from
                      > > >>>> source code and one generating the apps from the tests, I'd pick the
                      > > >>> second
                      > > >>>> and write the tests. AFAIK, neither is on offer. :-(
                      > > >>>>
                      > > >>>> Charlie
                      > > >>>>
                      > > >>>> On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
                      > > >>>> <lists@... <mailto:lists%40idiacomputing.com>
                      > > <mailto:lists%40idiacomputing.com>>wrote:
                      > > >>
                      > > >>>>
                      > > >>>>> **
                      > > >>>>>
                      > > >>>>>
                      > > >>>>> On 3/20/12 12:32 PM, M. Manca wrote:
                      > > >>>>>> In the embedded tool market now there are companies promoting their
                      > > >>>>>> (old) traditional products as useful to shorten the agile
                      > > >>> development
                      > > >>>>>> times. Their theory (obviously wrong) is that adding more SLOC or
                      > > >>>>>> function points to the software needed directly by the application
                      > > >>>>>> causes a cost increasing proportional to the ratio of the SLOCs
                      > > >>> fir the
                      > > >>>>>> (unit) tests and the application's SLOCs.
                      > > >>>>>>
                      > > >>>>>> So their are saying that auto generate unit test code (obviously
                      > > >> you
                      > > >>>>>> need existent code to test) guarantees a minor cost and the same
                      > > >>>> quality
                      > > >>>>>> of the agile unit test without their higher costs.
                      > > >>>>>>
                      > > >>>>>> I am sure that this is totally wrong my question is: anyone
                      > > >>> knows some
                      > > >>>>>> articles or materials about this argument (because I am sure
                      > > >>> that this
                      > > >>>>>> is not the 1st time someone said this) and articles to contest this
                      > > >>>> poor
                      > > >>>>>> vision about unit tests (more formally correct: developer tests
                      > > >>> in this
                      > > >>>>>> situation)?
                      > > >>>>>
                      > > >>>>> I wouldn't know where to start on writing such an article. How
                      > > can an
                      > > >>>>> automated tool know what the programmer intended?
                      > > >>>>>
                      > > >>>>> I looked at one such tool, years back, and all it did was pass
                      > > >>> nulls as
                      > > >>>>> each argument to a method. Since my code used the Null Object
                      > > >> Pattern,
                      > > >>>>> this was exactly contrary to my intent.
                      > > >>>>>
                      > > >>>>> - George
                      > > >>>>>
                      > > >>>>> --
                      > > >>>>> ----------------------------------------------------------
                      > > >>>>> * George Dinwiddie * http://blog.gdinwiddie.com
                      > > >>>>> Software Development http://www.idiacomputing.com
                      > > >>>>> Consultant and Coach http://www.agilemaryland.org
                      > > >>>>> ----------------------------------------------------------
                      > > >>>>>
                      > > >>>>>
                      > > >>>>>
                      > > >>>>
                      > > >>>>
                      > > >>>> [Non-text portions of this message have been removed]
                      > > >>>>
                      > > >>>>
                      > > >>>>
                      > > >>>> ------------------------------------
                      > > >>>>
                      > > >>>> To Post a message, send it to: extremeprogramming@...
                      > > <mailto:extremeprogramming%40eGroups.com>
                      > > >>> <mailto:extremeprogramming%40eGroups.com>
                      > > >>
                      > > >>>>
                      > > >>>> To Unsubscribe, send a blank message to:
                      > > >>>> extremeprogramming-unsubscribe@...
                      > > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                      > > >>> <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                      > > >>
                      > > >>>>
                      > > >>>> ad-free courtesy of objectmentor.comYahoo! Groups Links
                      > > >>>>
                      > > >>>>
                      > > >>>>
                      > > >>>>
                      > > >>>
                      > > >>> [Non-text portions of this message have been removed]
                      > > >>>
                      > > >>>
                      > > >>
                      > > >> [Non-text portions of this message have been removed]
                      > > >>
                      > > >>
                      > > >>
                      > > >
                      > > >
                      > > > [Non-text portions of this message have been removed]
                      > > >
                      > > >
                      > > >
                      > > > ------------------------------------
                      > > >
                      > > > To Post a message, send it to: extremeprogramming@...
                      > > <mailto:extremeprogramming%40eGroups.com>
                      > > >
                      > > > To Unsubscribe, send a blank message to:
                      > > extremeprogramming-unsubscribe@...
                      > > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                      > > >
                      > > > ad-free courtesy of objectmentor.comYahoo! Groups Links
                      > > >
                      > > >
                      > > >
                      > >
                      > >
                      >
                      > [Non-text portions of this message have been removed]
                      >
                      >



                      [Non-text portions of this message have been removed]
                    • JeffGrigg
                      Agitar s Agitator is an excellent tool. Probably the best one on the market. (If you want 2nd best, and also like dealing with an extremely pushy sales
                      Message 10 of 10 , Mar 20, 2012
                      • 0 Attachment
                        Agitar's Agitator is an excellent tool. Probably the best one on the market.

                        (If you want 2nd best, and also like dealing with an extremely pushy sales department, look at Jtest by Parasoft too. http://www.parasoft.com/jsp/products/jtest.jsp http://en.wikipedia.org/wiki/Jtest )

                        The best such tools can do is to predict what kinds of inputs will produce odd and potentially incorrect results -- such as exceptions. They're a lot better than a sharp stick in the face, when one wants to add tests to legacy code. But as an alternative to TDD, they just don't cut it. If used in addition to TDD, I might see some value. But you still have to put in a significant amount of work to analyze, understand, and properly respond to each of the many many many "test cases" they generate.


                        --- Daniel Wildt <dwildt@...> wrote:
                        > I know this one...
                        > http://www.agitar.com/solutions/products/automated_junit_generation.html
                      Your message has been successfully submitted and would be delivered to recipients shortly.