Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Tools to automatically generate "unit tests" source code

Expand Messages
  • George Dinwiddie
    ... I wouldn t know where to start on writing such an article. How can an automated tool know what the programmer intended? I looked at one such tool, years
    Message 1 of 10 , Mar 20 10:07 AM
    • 0 Attachment
      On 3/20/12 12:32 PM, M. Manca wrote:
      > In the embedded tool market now there are companies promoting their
      > (old) traditional products as useful to shorten the agile development
      > times. Their theory (obviously wrong) is that adding more SLOC or
      > function points to the software needed directly by the application
      > causes a cost increasing proportional to the ratio of the SLOCs fir the
      > (unit) tests and the application's SLOCs.
      >
      > So their are saying that auto generate unit test code (obviously you
      > need existent code to test) guarantees a minor cost and the same quality
      > of the agile unit test without their higher costs.
      >
      > I am sure that this is totally wrong my question is: anyone knows some
      > articles or materials about this argument (because I am sure that this
      > is not the 1st time someone said this) and articles to contest this poor
      > vision about unit tests (more formally correct: developer tests in this
      > situation)?

      I wouldn't know where to start on writing such an article. How can an
      automated tool know what the programmer intended?

      I looked at one such tool, years back, and all it did was pass nulls as
      each argument to a method. Since my code used the Null Object Pattern,
      this was exactly contrary to my intent.

      - George

      --
      ----------------------------------------------------------------------
      * George Dinwiddie * http://blog.gdinwiddie.com
      Software Development http://www.idiacomputing.com
      Consultant and Coach http://www.agilemaryland.org
      ----------------------------------------------------------------------
    • Charlie Poole
      My feeling is that test-generation *might* be useful in *some* applications after the code was developed using TDD. Some apps (fewer than many people think)
      Message 2 of 10 , Mar 20 10:23 AM
      • 0 Attachment
        My feeling is that test-generation *might* be useful in *some* applications
        after the code was developed using TDD.

        Some apps (fewer than many people think) need more testing after the code
        is developed using TDD. It needs a very sophisticated implementation to do
        this effectively. One such implemenation is the human brain. Conceivably,
        an app could use code analysis and numerical analysis to find test cases
        that would add value. As an alternative, one could use brute force to
        generate large numbers of such cases, but this isn't so satisfactory.

        My guess is that such an approach would be useful - if it existed - in less
        than 1% of the projects I've seen. A naive approach to test generation
        won't be useful anywhere but has the danger that it might pull people away
        from doing what they need to be doing: writing the tests themselves.

        If I had a choice between two magical apps, one generating tests from
        source code and one generating the apps from the tests, I'd pick the second
        and write the tests. AFAIK, neither is on offer. :-(

        Charlie

        On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
        <lists@...>wrote:

        > **
        >
        >
        > On 3/20/12 12:32 PM, M. Manca wrote:
        > > In the embedded tool market now there are companies promoting their
        > > (old) traditional products as useful to shorten the agile development
        > > times. Their theory (obviously wrong) is that adding more SLOC or
        > > function points to the software needed directly by the application
        > > causes a cost increasing proportional to the ratio of the SLOCs fir the
        > > (unit) tests and the application's SLOCs.
        > >
        > > So their are saying that auto generate unit test code (obviously you
        > > need existent code to test) guarantees a minor cost and the same quality
        > > of the agile unit test without their higher costs.
        > >
        > > I am sure that this is totally wrong my question is: anyone knows some
        > > articles or materials about this argument (because I am sure that this
        > > is not the 1st time someone said this) and articles to contest this poor
        > > vision about unit tests (more formally correct: developer tests in this
        > > situation)?
        >
        > I wouldn't know where to start on writing such an article. How can an
        > automated tool know what the programmer intended?
        >
        > I looked at one such tool, years back, and all it did was pass nulls as
        > each argument to a method. Since my code used the Null Object Pattern,
        > this was exactly contrary to my intent.
        >
        > - George
        >
        > --
        > ----------------------------------------------------------
        > * George Dinwiddie * http://blog.gdinwiddie.com
        > Software Development http://www.idiacomputing.com
        > Consultant and Coach http://www.agilemaryland.org
        > ----------------------------------------------------------
        >
        >
        >


        [Non-text portions of this message have been removed]
      • Daniel Wildt
        I know this one... http://www.agitar.com/solutions/products/automated_junit_generation.html -- Daniel Wildt (+55 51 99989030) - @dwildt
        Message 3 of 10 , Mar 20 10:41 AM
        • 0 Attachment
          I know this one...
          http://www.agitar.com/solutions/products/automated_junit_generation.html


          -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>


          On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...>wrote:

          > My feeling is that test-generation *might* be useful in *some* applications
          > after the code was developed using TDD.
          >
          > Some apps (fewer than many people think) need more testing after the code
          > is developed using TDD. It needs a very sophisticated implementation to do
          > this effectively. One such implemenation is the human brain. Conceivably,
          > an app could use code analysis and numerical analysis to find test cases
          > that would add value. As an alternative, one could use brute force to
          > generate large numbers of such cases, but this isn't so satisfactory.
          >
          > My guess is that such an approach would be useful - if it existed - in less
          > than 1% of the projects I've seen. A naive approach to test generation
          > won't be useful anywhere but has the danger that it might pull people away
          > from doing what they need to be doing: writing the tests themselves.
          >
          > If I had a choice between two magical apps, one generating tests from
          > source code and one generating the apps from the tests, I'd pick the second
          > and write the tests. AFAIK, neither is on offer. :-(
          >
          > Charlie
          >
          > On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
          > <lists@...>wrote:
          >
          > > **
          > >
          > >
          > > On 3/20/12 12:32 PM, M. Manca wrote:
          > > > In the embedded tool market now there are companies promoting their
          > > > (old) traditional products as useful to shorten the agile development
          > > > times. Their theory (obviously wrong) is that adding more SLOC or
          > > > function points to the software needed directly by the application
          > > > causes a cost increasing proportional to the ratio of the SLOCs fir the
          > > > (unit) tests and the application's SLOCs.
          > > >
          > > > So their are saying that auto generate unit test code (obviously you
          > > > need existent code to test) guarantees a minor cost and the same
          > quality
          > > > of the agile unit test without their higher costs.
          > > >
          > > > I am sure that this is totally wrong my question is: anyone knows some
          > > > articles or materials about this argument (because I am sure that this
          > > > is not the 1st time someone said this) and articles to contest this
          > poor
          > > > vision about unit tests (more formally correct: developer tests in this
          > > > situation)?
          > >
          > > I wouldn't know where to start on writing such an article. How can an
          > > automated tool know what the programmer intended?
          > >
          > > I looked at one such tool, years back, and all it did was pass nulls as
          > > each argument to a method. Since my code used the Null Object Pattern,
          > > this was exactly contrary to my intent.
          > >
          > > - George
          > >
          > > --
          > > ----------------------------------------------------------
          > > * George Dinwiddie * http://blog.gdinwiddie.com
          > > Software Development http://www.idiacomputing.com
          > > Consultant and Coach http://www.agilemaryland.org
          > > ----------------------------------------------------------
          > >
          > >
          > >
          >
          >
          > [Non-text portions of this message have been removed]
          >
          >
          >
          > ------------------------------------
          >
          > To Post a message, send it to: extremeprogramming@...
          >
          > To Unsubscribe, send a blank message to:
          > extremeprogramming-unsubscribe@...
          >
          > ad-free courtesy of objectmentor.comYahoo! Groups Links
          >
          >
          >
          >


          [Non-text portions of this message have been removed]
        • M. Manca
          Il 20/03/2012 18:41, Daniel Wildt ha scritto: I need to correct myself: when I wrote article I was meaning a magazine article not a software tool or product.
          Message 4 of 10 , Mar 20 11:36 AM
          • 0 Attachment
            Il 20/03/2012 18:41, Daniel Wildt ha scritto:
            I need to correct myself: when I wrote "article" I was meaning a
            magazine article not a software tool or product.

            So I would find to try some documentation favorable and contrary to the
            automatic unit test generation.

            My opinion is that a unit test has to be written before the code to test
            in a TDD or BDD fashion.
            The advantage is embedded in the order of things, writing unit tests
            after the source code is just a new form of developer tests, writing the
            test before is a way to transform a written requirement in an executable
            requirement and this is the very big advantage.

            I feel very natural to write the test before the code that implements
            the behavior.
            >
            >
            > I know this one...
            > http://www.agitar.com/solutions/products/automated_junit_generation.html
            >
            > -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>
            >
            > On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...
            > <mailto:charliepoole%40gmail.com>>wrote:
            >
            > > My feeling is that test-generation *might* be useful in *some*
            > applications
            > > after the code was developed using TDD.
            > >
            > > Some apps (fewer than many people think) need more testing after the
            > code
            > > is developed using TDD. It needs a very sophisticated implementation
            > to do
            > > this effectively. One such implemenation is the human brain.
            > Conceivably,
            > > an app could use code analysis and numerical analysis to find test cases
            > > that would add value. As an alternative, one could use brute force to
            > > generate large numbers of such cases, but this isn't so satisfactory.
            > >
            > > My guess is that such an approach would be useful - if it existed -
            > in less
            > > than 1% of the projects I've seen. A naive approach to test generation
            > > won't be useful anywhere but has the danger that it might pull
            > people away
            > > from doing what they need to be doing: writing the tests themselves.
            > >
            > > If I had a choice between two magical apps, one generating tests from
            > > source code and one generating the apps from the tests, I'd pick the
            > second
            > > and write the tests. AFAIK, neither is on offer. :-(
            > >
            > > Charlie
            > >
            > > On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
            > > <lists@... <mailto:lists%40idiacomputing.com>>wrote:
            > >
            > > > **
            > > >
            > > >
            > > > On 3/20/12 12:32 PM, M. Manca wrote:
            > > > > In the embedded tool market now there are companies promoting their
            > > > > (old) traditional products as useful to shorten the agile
            > development
            > > > > times. Their theory (obviously wrong) is that adding more SLOC or
            > > > > function points to the software needed directly by the application
            > > > > causes a cost increasing proportional to the ratio of the SLOCs
            > fir the
            > > > > (unit) tests and the application's SLOCs.
            > > > >
            > > > > So their are saying that auto generate unit test code (obviously you
            > > > > need existent code to test) guarantees a minor cost and the same
            > > quality
            > > > > of the agile unit test without their higher costs.
            > > > >
            > > > > I am sure that this is totally wrong my question is: anyone
            > knows some
            > > > > articles or materials about this argument (because I am sure
            > that this
            > > > > is not the 1st time someone said this) and articles to contest this
            > > poor
            > > > > vision about unit tests (more formally correct: developer tests
            > in this
            > > > > situation)?
            > > >
            > > > I wouldn't know where to start on writing such an article. How can an
            > > > automated tool know what the programmer intended?
            > > >
            > > > I looked at one such tool, years back, and all it did was pass
            > nulls as
            > > > each argument to a method. Since my code used the Null Object Pattern,
            > > > this was exactly contrary to my intent.
            > > >
            > > > - George
            > > >
            > > > --
            > > > ----------------------------------------------------------
            > > > * George Dinwiddie * http://blog.gdinwiddie.com
            > > > Software Development http://www.idiacomputing.com
            > > > Consultant and Coach http://www.agilemaryland.org
            > > > ----------------------------------------------------------
            > > >
            > > >
            > > >
            > >
            > >
            > > [Non-text portions of this message have been removed]
            > >
            > >
            > >
            > > ------------------------------------
            > >
            > > To Post a message, send it to: extremeprogramming@...
            > <mailto:extremeprogramming%40eGroups.com>
            > >
            > > To Unsubscribe, send a blank message to:
            > > extremeprogramming-unsubscribe@...
            > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
            > >
            > > ad-free courtesy of objectmentor.comYahoo! Groups Links
            > >
            > >
            > >
            > >
            >
            > [Non-text portions of this message have been removed]
            >
            >



            [Non-text portions of this message have been removed]
          • Adam Sroka
            I have heard the argument before and I have seen some of the tools that attempt to do this. I haven t delved deeply into it, but my initial impression is that
            Message 5 of 10 , Mar 20 11:54 AM
            • 0 Attachment
              I have heard the argument before and I have seen some of the tools that
              attempt to do this. I haven't delved deeply into it, but my initial
              impression is that these tools tend to create a bunch of brittle,
              uncommunicative tests. The tools lack the domain knowledge and intelligence
              necessary to craft well-written tests with meaningful assertions that serve
              as documentation of the code. If the only quality you care about in testing
              is coverage then you might find them useful.

              On Tue, Mar 20, 2012 at 11:36 AM, M. Manca <m.manca@...>wrote:

              > **
              >
              >
              > Il 20/03/2012 18:41, Daniel Wildt ha scritto:
              > I need to correct myself: when I wrote "article" I was meaning a
              > magazine article not a software tool or product.
              >
              > So I would find to try some documentation favorable and contrary to the
              > automatic unit test generation.
              >
              > My opinion is that a unit test has to be written before the code to test
              > in a TDD or BDD fashion.
              > The advantage is embedded in the order of things, writing unit tests
              > after the source code is just a new form of developer tests, writing the
              > test before is a way to transform a written requirement in an executable
              > requirement and this is the very big advantage.
              >
              > I feel very natural to write the test before the code that implements
              > the behavior.
              >
              > >
              > >
              > > I know this one...
              > > http://www.agitar.com/solutions/products/automated_junit_generation.html
              > >
              > > -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>
              > >
              > > On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...
              > > <mailto:charliepoole%40gmail.com>>wrote:
              >
              > >
              > > > My feeling is that test-generation *might* be useful in *some*
              > > applications
              > > > after the code was developed using TDD.
              > > >
              > > > Some apps (fewer than many people think) need more testing after the
              > > code
              > > > is developed using TDD. It needs a very sophisticated implementation
              > > to do
              > > > this effectively. One such implemenation is the human brain.
              > > Conceivably,
              > > > an app could use code analysis and numerical analysis to find test
              > cases
              > > > that would add value. As an alternative, one could use brute force to
              > > > generate large numbers of such cases, but this isn't so satisfactory.
              > > >
              > > > My guess is that such an approach would be useful - if it existed -
              > > in less
              > > > than 1% of the projects I've seen. A naive approach to test generation
              > > > won't be useful anywhere but has the danger that it might pull
              > > people away
              > > > from doing what they need to be doing: writing the tests themselves.
              > > >
              > > > If I had a choice between two magical apps, one generating tests from
              > > > source code and one generating the apps from the tests, I'd pick the
              > > second
              > > > and write the tests. AFAIK, neither is on offer. :-(
              > > >
              > > > Charlie
              > > >
              > > > On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
              > > > <lists@... <mailto:lists%40idiacomputing.com>>wrote:
              >
              > > >
              > > > > **
              > > > >
              > > > >
              > > > > On 3/20/12 12:32 PM, M. Manca wrote:
              > > > > > In the embedded tool market now there are companies promoting their
              > > > > > (old) traditional products as useful to shorten the agile
              > > development
              > > > > > times. Their theory (obviously wrong) is that adding more SLOC or
              > > > > > function points to the software needed directly by the application
              > > > > > causes a cost increasing proportional to the ratio of the SLOCs
              > > fir the
              > > > > > (unit) tests and the application's SLOCs.
              > > > > >
              > > > > > So their are saying that auto generate unit test code (obviously
              > you
              > > > > > need existent code to test) guarantees a minor cost and the same
              > > > quality
              > > > > > of the agile unit test without their higher costs.
              > > > > >
              > > > > > I am sure that this is totally wrong my question is: anyone
              > > knows some
              > > > > > articles or materials about this argument (because I am sure
              > > that this
              > > > > > is not the 1st time someone said this) and articles to contest this
              > > > poor
              > > > > > vision about unit tests (more formally correct: developer tests
              > > in this
              > > > > > situation)?
              > > > >
              > > > > I wouldn't know where to start on writing such an article. How can an
              > > > > automated tool know what the programmer intended?
              > > > >
              > > > > I looked at one such tool, years back, and all it did was pass
              > > nulls as
              > > > > each argument to a method. Since my code used the Null Object
              > Pattern,
              > > > > this was exactly contrary to my intent.
              > > > >
              > > > > - George
              > > > >
              > > > > --
              > > > > ----------------------------------------------------------
              > > > > * George Dinwiddie * http://blog.gdinwiddie.com
              > > > > Software Development http://www.idiacomputing.com
              > > > > Consultant and Coach http://www.agilemaryland.org
              > > > > ----------------------------------------------------------
              > > > >
              > > > >
              > > > >
              > > >
              > > >
              > > > [Non-text portions of this message have been removed]
              > > >
              > > >
              > > >
              > > > ------------------------------------
              > > >
              > > > To Post a message, send it to: extremeprogramming@...
              > > <mailto:extremeprogramming%40eGroups.com>
              >
              > > >
              > > > To Unsubscribe, send a blank message to:
              > > > extremeprogramming-unsubscribe@...
              > > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
              >
              > > >
              > > > ad-free courtesy of objectmentor.comYahoo! Groups Links
              > > >
              > > >
              > > >
              > > >
              > >
              > > [Non-text portions of this message have been removed]
              > >
              > >
              >
              > [Non-text portions of this message have been removed]
              >
              >
              >


              [Non-text portions of this message have been removed]
            • james@renaissancesoftware.net
              Funny, I had a client using tools like this last week. I agree with Adam that these tests are of little value. Aside from helping achieve a metric (that
              Message 6 of 10 , Mar 20 12:28 PM
              • 0 Attachment
                Funny, I had a client using tools like this last week. I agree with Adam that these tests are of little value. Aside from helping achieve a metric (that should not be the goal anyway), these generated tests might be helpful when making sure you understand the consequences when changing legacy code. They cannot provide the other benefits from hand crafted tests and TDD.

                thanks, James

                --------------------------------------------------------------------------------------------
                James Grenning Author of TDD for Embedded C
                www.renaissancesoftware.net http://pragprog.com/titles/jgade/
                www.renaissancesoftware.net/blog
                www.twitter.com/jwgrenning


                On Mar 20, 2012, at 1:54 PM, Adam Sroka wrote:

                > I have heard the argument before and I have seen some of the tools that
                > attempt to do this. I haven't delved deeply into it, but my initial
                > impression is that these tools tend to create a bunch of brittle,
                > uncommunicative tests. The tools lack the domain knowledge and intelligence
                > necessary to craft well-written tests with meaningful assertions that serve
                > as documentation of the code. If the only quality you care about in testing
                > is coverage then you might find them useful.
                >
                > On Tue, Mar 20, 2012 at 11:36 AM, M. Manca <m.manca@...>wrote:
                >
                >> **
                >>
                >>
                >> Il 20/03/2012 18:41, Daniel Wildt ha scritto:
                >> I need to correct myself: when I wrote "article" I was meaning a
                >> magazine article not a software tool or product.
                >>
                >> So I would find to try some documentation favorable and contrary to the
                >> automatic unit test generation.
                >>
                >> My opinion is that a unit test has to be written before the code to test
                >> in a TDD or BDD fashion.
                >> The advantage is embedded in the order of things, writing unit tests
                >> after the source code is just a new form of developer tests, writing the
                >> test before is a way to transform a written requirement in an executable
                >> requirement and this is the very big advantage.
                >>
                >> I feel very natural to write the test before the code that implements
                >> the behavior.
                >>
                >>>
                >>>
                >>> I know this one...
                >>> http://www.agitar.com/solutions/products/automated_junit_generation.html
                >>>
                >>> -- Daniel Wildt (+55 51 99989030) - @dwildt <http://twitter.com/dwildt>
                >>>
                >>> On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole <charliepoole@...
                >>> <mailto:charliepoole%40gmail.com>>wrote:
                >>
                >>>
                >>>> My feeling is that test-generation *might* be useful in *some*
                >>> applications
                >>>> after the code was developed using TDD.
                >>>>
                >>>> Some apps (fewer than many people think) need more testing after the
                >>> code
                >>>> is developed using TDD. It needs a very sophisticated implementation
                >>> to do
                >>>> this effectively. One such implemenation is the human brain.
                >>> Conceivably,
                >>>> an app could use code analysis and numerical analysis to find test
                >> cases
                >>>> that would add value. As an alternative, one could use brute force to
                >>>> generate large numbers of such cases, but this isn't so satisfactory.
                >>>>
                >>>> My guess is that such an approach would be useful - if it existed -
                >>> in less
                >>>> than 1% of the projects I've seen. A naive approach to test generation
                >>>> won't be useful anywhere but has the danger that it might pull
                >>> people away
                >>>> from doing what they need to be doing: writing the tests themselves.
                >>>>
                >>>> If I had a choice between two magical apps, one generating tests from
                >>>> source code and one generating the apps from the tests, I'd pick the
                >>> second
                >>>> and write the tests. AFAIK, neither is on offer. :-(
                >>>>
                >>>> Charlie
                >>>>
                >>>> On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
                >>>> <lists@... <mailto:lists%40idiacomputing.com>>wrote:
                >>
                >>>>
                >>>>> **
                >>>>>
                >>>>>
                >>>>> On 3/20/12 12:32 PM, M. Manca wrote:
                >>>>>> In the embedded tool market now there are companies promoting their
                >>>>>> (old) traditional products as useful to shorten the agile
                >>> development
                >>>>>> times. Their theory (obviously wrong) is that adding more SLOC or
                >>>>>> function points to the software needed directly by the application
                >>>>>> causes a cost increasing proportional to the ratio of the SLOCs
                >>> fir the
                >>>>>> (unit) tests and the application's SLOCs.
                >>>>>>
                >>>>>> So their are saying that auto generate unit test code (obviously
                >> you
                >>>>>> need existent code to test) guarantees a minor cost and the same
                >>>> quality
                >>>>>> of the agile unit test without their higher costs.
                >>>>>>
                >>>>>> I am sure that this is totally wrong my question is: anyone
                >>> knows some
                >>>>>> articles or materials about this argument (because I am sure
                >>> that this
                >>>>>> is not the 1st time someone said this) and articles to contest this
                >>>> poor
                >>>>>> vision about unit tests (more formally correct: developer tests
                >>> in this
                >>>>>> situation)?
                >>>>>
                >>>>> I wouldn't know where to start on writing such an article. How can an
                >>>>> automated tool know what the programmer intended?
                >>>>>
                >>>>> I looked at one such tool, years back, and all it did was pass
                >>> nulls as
                >>>>> each argument to a method. Since my code used the Null Object
                >> Pattern,
                >>>>> this was exactly contrary to my intent.
                >>>>>
                >>>>> - George
                >>>>>
                >>>>> --
                >>>>> ----------------------------------------------------------
                >>>>> * George Dinwiddie * http://blog.gdinwiddie.com
                >>>>> Software Development http://www.idiacomputing.com
                >>>>> Consultant and Coach http://www.agilemaryland.org
                >>>>> ----------------------------------------------------------
                >>>>>
                >>>>>
                >>>>>
                >>>>
                >>>>
                >>>> [Non-text portions of this message have been removed]
                >>>>
                >>>>
                >>>>
                >>>> ------------------------------------
                >>>>
                >>>> To Post a message, send it to: extremeprogramming@...
                >>> <mailto:extremeprogramming%40eGroups.com>
                >>
                >>>>
                >>>> To Unsubscribe, send a blank message to:
                >>>> extremeprogramming-unsubscribe@...
                >>> <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                >>
                >>>>
                >>>> ad-free courtesy of objectmentor.comYahoo! Groups Links
                >>>>
                >>>>
                >>>>
                >>>>
                >>>
                >>> [Non-text portions of this message have been removed]
                >>>
                >>>
                >>
                >> [Non-text portions of this message have been removed]
                >>
                >>
                >>
                >
                >
                > [Non-text portions of this message have been removed]
                >
                >
                >
                > ------------------------------------
                >
                > To Post a message, send it to: extremeprogramming@...
                >
                > To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
                >
                > ad-free courtesy of objectmentor.comYahoo! Groups Links
                >
                >
                >
              • M. Manca
                ... So we all agree that these tools have little value. This is a good thing, now how can we communicate this idea to the embedded developers community?
                Message 7 of 10 , Mar 20 12:42 PM
                • 0 Attachment
                  Il 20/03/2012 20:28, james@... ha scritto:
                  >
                  >
                  > Funny, I had a client using tools like this last week. I agree with
                  > Adam that these tests are of little value. Aside from helping achieve
                  > a metric (that should not be the goal anyway), these generated tests
                  > might be helpful when making sure you understand the consequences when
                  > changing legacy code. They cannot provide the other benefits from hand
                  > crafted tests and TDD.
                  >
                  > thanks, James
                  >
                  So we all agree that these tools have little value.
                  This is a good thing, now how can we communicate this idea to the
                  embedded developers community? Because if no one say that these tools
                  have no value the idea that will pass is that they help to deliver
                  better software.
                  >
                  >
                  > ----------------------------------------------------------
                  > James Grenning Author of TDD for Embedded C
                  > www.renaissancesoftware.net http://pragprog.com/titles/jgade/
                  > www.renaissancesoftware.net/blog
                  > www.twitter.com/jwgrenning
                  >
                  > On Mar 20, 2012, at 1:54 PM, Adam Sroka wrote:
                  >
                  > > I have heard the argument before and I have seen some of the tools that
                  > > attempt to do this. I haven't delved deeply into it, but my initial
                  > > impression is that these tools tend to create a bunch of brittle,
                  > > uncommunicative tests. The tools lack the domain knowledge and
                  > intelligence
                  > > necessary to craft well-written tests with meaningful assertions
                  > that serve
                  > > as documentation of the code. If the only quality you care about in
                  > testing
                  > > is coverage then you might find them useful.
                  > >
                  > > On Tue, Mar 20, 2012 at 11:36 AM, M. Manca
                  > <m.manca@...
                  > <mailto:m.manca%40micronengineering.it>>wrote:
                  > >
                  > >> **
                  > >>
                  > >>
                  > >> Il 20/03/2012 18:41, Daniel Wildt ha scritto:
                  > >> I need to correct myself: when I wrote "article" I was meaning a
                  > >> magazine article not a software tool or product.
                  > >>
                  > >> So I would find to try some documentation favorable and contrary to the
                  > >> automatic unit test generation.
                  > >>
                  > >> My opinion is that a unit test has to be written before the code to
                  > test
                  > >> in a TDD or BDD fashion.
                  > >> The advantage is embedded in the order of things, writing unit tests
                  > >> after the source code is just a new form of developer tests,
                  > writing the
                  > >> test before is a way to transform a written requirement in an
                  > executable
                  > >> requirement and this is the very big advantage.
                  > >>
                  > >> I feel very natural to write the test before the code that implements
                  > >> the behavior.
                  > >>
                  > >>>
                  > >>>
                  > >>> I know this one...
                  > >>>
                  > http://www.agitar.com/solutions/products/automated_junit_generation.html
                  > >>>
                  > >>> -- Daniel Wildt (+55 51 99989030) - @dwildt
                  > <http://twitter.com/dwildt>
                  > >>>
                  > >>> On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole
                  > <charliepoole@... <mailto:charliepoole%40gmail.com>
                  > >>> <mailto:charliepoole%40gmail.com>>wrote:
                  > >>
                  > >>>
                  > >>>> My feeling is that test-generation *might* be useful in *some*
                  > >>> applications
                  > >>>> after the code was developed using TDD.
                  > >>>>
                  > >>>> Some apps (fewer than many people think) need more testing after the
                  > >>> code
                  > >>>> is developed using TDD. It needs a very sophisticated implementation
                  > >>> to do
                  > >>>> this effectively. One such implemenation is the human brain.
                  > >>> Conceivably,
                  > >>>> an app could use code analysis and numerical analysis to find test
                  > >> cases
                  > >>>> that would add value. As an alternative, one could use brute force to
                  > >>>> generate large numbers of such cases, but this isn't so satisfactory.
                  > >>>>
                  > >>>> My guess is that such an approach would be useful - if it existed -
                  > >>> in less
                  > >>>> than 1% of the projects I've seen. A naive approach to test
                  > generation
                  > >>>> won't be useful anywhere but has the danger that it might pull
                  > >>> people away
                  > >>>> from doing what they need to be doing: writing the tests themselves.
                  > >>>>
                  > >>>> If I had a choice between two magical apps, one generating tests from
                  > >>>> source code and one generating the apps from the tests, I'd pick the
                  > >>> second
                  > >>>> and write the tests. AFAIK, neither is on offer. :-(
                  > >>>>
                  > >>>> Charlie
                  > >>>>
                  > >>>> On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
                  > >>>> <lists@... <mailto:lists%40idiacomputing.com>
                  > <mailto:lists%40idiacomputing.com>>wrote:
                  > >>
                  > >>>>
                  > >>>>> **
                  > >>>>>
                  > >>>>>
                  > >>>>> On 3/20/12 12:32 PM, M. Manca wrote:
                  > >>>>>> In the embedded tool market now there are companies promoting their
                  > >>>>>> (old) traditional products as useful to shorten the agile
                  > >>> development
                  > >>>>>> times. Their theory (obviously wrong) is that adding more SLOC or
                  > >>>>>> function points to the software needed directly by the application
                  > >>>>>> causes a cost increasing proportional to the ratio of the SLOCs
                  > >>> fir the
                  > >>>>>> (unit) tests and the application's SLOCs.
                  > >>>>>>
                  > >>>>>> So their are saying that auto generate unit test code (obviously
                  > >> you
                  > >>>>>> need existent code to test) guarantees a minor cost and the same
                  > >>>> quality
                  > >>>>>> of the agile unit test without their higher costs.
                  > >>>>>>
                  > >>>>>> I am sure that this is totally wrong my question is: anyone
                  > >>> knows some
                  > >>>>>> articles or materials about this argument (because I am sure
                  > >>> that this
                  > >>>>>> is not the 1st time someone said this) and articles to contest this
                  > >>>> poor
                  > >>>>>> vision about unit tests (more formally correct: developer tests
                  > >>> in this
                  > >>>>>> situation)?
                  > >>>>>
                  > >>>>> I wouldn't know where to start on writing such an article. How
                  > can an
                  > >>>>> automated tool know what the programmer intended?
                  > >>>>>
                  > >>>>> I looked at one such tool, years back, and all it did was pass
                  > >>> nulls as
                  > >>>>> each argument to a method. Since my code used the Null Object
                  > >> Pattern,
                  > >>>>> this was exactly contrary to my intent.
                  > >>>>>
                  > >>>>> - George
                  > >>>>>
                  > >>>>> --
                  > >>>>> ----------------------------------------------------------
                  > >>>>> * George Dinwiddie * http://blog.gdinwiddie.com
                  > >>>>> Software Development http://www.idiacomputing.com
                  > >>>>> Consultant and Coach http://www.agilemaryland.org
                  > >>>>> ----------------------------------------------------------
                  > >>>>>
                  > >>>>>
                  > >>>>>
                  > >>>>
                  > >>>>
                  > >>>> [Non-text portions of this message have been removed]
                  > >>>>
                  > >>>>
                  > >>>>
                  > >>>> ------------------------------------
                  > >>>>
                  > >>>> To Post a message, send it to: extremeprogramming@...
                  > <mailto:extremeprogramming%40eGroups.com>
                  > >>> <mailto:extremeprogramming%40eGroups.com>
                  > >>
                  > >>>>
                  > >>>> To Unsubscribe, send a blank message to:
                  > >>>> extremeprogramming-unsubscribe@...
                  > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                  > >>> <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                  > >>
                  > >>>>
                  > >>>> ad-free courtesy of objectmentor.comYahoo! Groups Links
                  > >>>>
                  > >>>>
                  > >>>>
                  > >>>>
                  > >>>
                  > >>> [Non-text portions of this message have been removed]
                  > >>>
                  > >>>
                  > >>
                  > >> [Non-text portions of this message have been removed]
                  > >>
                  > >>
                  > >>
                  > >
                  > >
                  > > [Non-text portions of this message have been removed]
                  > >
                  > >
                  > >
                  > > ------------------------------------
                  > >
                  > > To Post a message, send it to: extremeprogramming@...
                  > <mailto:extremeprogramming%40eGroups.com>
                  > >
                  > > To Unsubscribe, send a blank message to:
                  > extremeprogramming-unsubscribe@...
                  > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                  > >
                  > > ad-free courtesy of objectmentor.comYahoo! Groups Links
                  > >
                  > >
                  > >
                  >
                  >



                  [Non-text portions of this message have been removed]
                • james@renaissancesoftware.net
                  I might break the news to some of them at my Embedded Systems Conference sessions next week. ... [Non-text portions of this message have been removed]
                  Message 8 of 10 , Mar 20 1:23 PM
                  • 0 Attachment
                    I might break the news to some of them at my Embedded Systems Conference sessions next week.


                    On Mar 20, 2012, at 2:42 PM, M. Manca wrote:

                    > Il 20/03/2012 20:28, james@... ha scritto:
                    > >
                    > >
                    > > Funny, I had a client using tools like this last week. I agree with
                    > > Adam that these tests are of little value. Aside from helping achieve
                    > > a metric (that should not be the goal anyway), these generated tests
                    > > might be helpful when making sure you understand the consequences when
                    > > changing legacy code. They cannot provide the other benefits from hand
                    > > crafted tests and TDD.
                    > >
                    > > thanks, James
                    > >
                    > So we all agree that these tools have little value.
                    > This is a good thing, now how can we communicate this idea to the
                    > embedded developers community? Because if no one say that these tools
                    > have no value the idea that will pass is that they help to deliver
                    > better software.
                    > >
                    > >
                    > > ----------------------------------------------------------
                    > > James Grenning Author of TDD for Embedded C
                    > > www.renaissancesoftware.net http://pragprog.com/titles/jgade/
                    > > www.renaissancesoftware.net/blog
                    > > www.twitter.com/jwgrenning
                    > >
                    > > On Mar 20, 2012, at 1:54 PM, Adam Sroka wrote:
                    > >
                    > > > I have heard the argument before and I have seen some of the tools that
                    > > > attempt to do this. I haven't delved deeply into it, but my initial
                    > > > impression is that these tools tend to create a bunch of brittle,
                    > > > uncommunicative tests. The tools lack the domain knowledge and
                    > > intelligence
                    > > > necessary to craft well-written tests with meaningful assertions
                    > > that serve
                    > > > as documentation of the code. If the only quality you care about in
                    > > testing
                    > > > is coverage then you might find them useful.
                    > > >
                    > > > On Tue, Mar 20, 2012 at 11:36 AM, M. Manca
                    > > <m.manca@...
                    > > <mailto:m.manca%40micronengineering.it>>wrote:
                    > > >
                    > > >> **
                    > > >>
                    > > >>
                    > > >> Il 20/03/2012 18:41, Daniel Wildt ha scritto:
                    > > >> I need to correct myself: when I wrote "article" I was meaning a
                    > > >> magazine article not a software tool or product.
                    > > >>
                    > > >> So I would find to try some documentation favorable and contrary to the
                    > > >> automatic unit test generation.
                    > > >>
                    > > >> My opinion is that a unit test has to be written before the code to
                    > > test
                    > > >> in a TDD or BDD fashion.
                    > > >> The advantage is embedded in the order of things, writing unit tests
                    > > >> after the source code is just a new form of developer tests,
                    > > writing the
                    > > >> test before is a way to transform a written requirement in an
                    > > executable
                    > > >> requirement and this is the very big advantage.
                    > > >>
                    > > >> I feel very natural to write the test before the code that implements
                    > > >> the behavior.
                    > > >>
                    > > >>>
                    > > >>>
                    > > >>> I know this one...
                    > > >>>
                    > > http://www.agitar.com/solutions/products/automated_junit_generation.html
                    > > >>>
                    > > >>> -- Daniel Wildt (+55 51 99989030) - @dwildt
                    > > <http://twitter.com/dwildt>
                    > > >>>
                    > > >>> On Tue, Mar 20, 2012 at 2:23 PM, Charlie Poole
                    > > <charliepoole@... <mailto:charliepoole%40gmail.com>
                    > > >>> <mailto:charliepoole%40gmail.com>>wrote:
                    > > >>
                    > > >>>
                    > > >>>> My feeling is that test-generation *might* be useful in *some*
                    > > >>> applications
                    > > >>>> after the code was developed using TDD.
                    > > >>>>
                    > > >>>> Some apps (fewer than many people think) need more testing after the
                    > > >>> code
                    > > >>>> is developed using TDD. It needs a very sophisticated implementation
                    > > >>> to do
                    > > >>>> this effectively. One such implemenation is the human brain.
                    > > >>> Conceivably,
                    > > >>>> an app could use code analysis and numerical analysis to find test
                    > > >> cases
                    > > >>>> that would add value. As an alternative, one could use brute force to
                    > > >>>> generate large numbers of such cases, but this isn't so satisfactory.
                    > > >>>>
                    > > >>>> My guess is that such an approach would be useful - if it existed -
                    > > >>> in less
                    > > >>>> than 1% of the projects I've seen. A naive approach to test
                    > > generation
                    > > >>>> won't be useful anywhere but has the danger that it might pull
                    > > >>> people away
                    > > >>>> from doing what they need to be doing: writing the tests themselves.
                    > > >>>>
                    > > >>>> If I had a choice between two magical apps, one generating tests from
                    > > >>>> source code and one generating the apps from the tests, I'd pick the
                    > > >>> second
                    > > >>>> and write the tests. AFAIK, neither is on offer. :-(
                    > > >>>>
                    > > >>>> Charlie
                    > > >>>>
                    > > >>>> On Tue, Mar 20, 2012 at 10:07 AM, George Dinwiddie
                    > > >>>> <lists@... <mailto:lists%40idiacomputing.com>
                    > > <mailto:lists%40idiacomputing.com>>wrote:
                    > > >>
                    > > >>>>
                    > > >>>>> **
                    > > >>>>>
                    > > >>>>>
                    > > >>>>> On 3/20/12 12:32 PM, M. Manca wrote:
                    > > >>>>>> In the embedded tool market now there are companies promoting their
                    > > >>>>>> (old) traditional products as useful to shorten the agile
                    > > >>> development
                    > > >>>>>> times. Their theory (obviously wrong) is that adding more SLOC or
                    > > >>>>>> function points to the software needed directly by the application
                    > > >>>>>> causes a cost increasing proportional to the ratio of the SLOCs
                    > > >>> fir the
                    > > >>>>>> (unit) tests and the application's SLOCs.
                    > > >>>>>>
                    > > >>>>>> So their are saying that auto generate unit test code (obviously
                    > > >> you
                    > > >>>>>> need existent code to test) guarantees a minor cost and the same
                    > > >>>> quality
                    > > >>>>>> of the agile unit test without their higher costs.
                    > > >>>>>>
                    > > >>>>>> I am sure that this is totally wrong my question is: anyone
                    > > >>> knows some
                    > > >>>>>> articles or materials about this argument (because I am sure
                    > > >>> that this
                    > > >>>>>> is not the 1st time someone said this) and articles to contest this
                    > > >>>> poor
                    > > >>>>>> vision about unit tests (more formally correct: developer tests
                    > > >>> in this
                    > > >>>>>> situation)?
                    > > >>>>>
                    > > >>>>> I wouldn't know where to start on writing such an article. How
                    > > can an
                    > > >>>>> automated tool know what the programmer intended?
                    > > >>>>>
                    > > >>>>> I looked at one such tool, years back, and all it did was pass
                    > > >>> nulls as
                    > > >>>>> each argument to a method. Since my code used the Null Object
                    > > >> Pattern,
                    > > >>>>> this was exactly contrary to my intent.
                    > > >>>>>
                    > > >>>>> - George
                    > > >>>>>
                    > > >>>>> --
                    > > >>>>> ----------------------------------------------------------
                    > > >>>>> * George Dinwiddie * http://blog.gdinwiddie.com
                    > > >>>>> Software Development http://www.idiacomputing.com
                    > > >>>>> Consultant and Coach http://www.agilemaryland.org
                    > > >>>>> ----------------------------------------------------------
                    > > >>>>>
                    > > >>>>>
                    > > >>>>>
                    > > >>>>
                    > > >>>>
                    > > >>>> [Non-text portions of this message have been removed]
                    > > >>>>
                    > > >>>>
                    > > >>>>
                    > > >>>> ------------------------------------
                    > > >>>>
                    > > >>>> To Post a message, send it to: extremeprogramming@...
                    > > <mailto:extremeprogramming%40eGroups.com>
                    > > >>> <mailto:extremeprogramming%40eGroups.com>
                    > > >>
                    > > >>>>
                    > > >>>> To Unsubscribe, send a blank message to:
                    > > >>>> extremeprogramming-unsubscribe@...
                    > > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                    > > >>> <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                    > > >>
                    > > >>>>
                    > > >>>> ad-free courtesy of objectmentor.comYahoo! Groups Links
                    > > >>>>
                    > > >>>>
                    > > >>>>
                    > > >>>>
                    > > >>>
                    > > >>> [Non-text portions of this message have been removed]
                    > > >>>
                    > > >>>
                    > > >>
                    > > >> [Non-text portions of this message have been removed]
                    > > >>
                    > > >>
                    > > >>
                    > > >
                    > > >
                    > > > [Non-text portions of this message have been removed]
                    > > >
                    > > >
                    > > >
                    > > > ------------------------------------
                    > > >
                    > > > To Post a message, send it to: extremeprogramming@...
                    > > <mailto:extremeprogramming%40eGroups.com>
                    > > >
                    > > > To Unsubscribe, send a blank message to:
                    > > extremeprogramming-unsubscribe@...
                    > > <mailto:extremeprogramming-unsubscribe%40eGroups.com>
                    > > >
                    > > > ad-free courtesy of objectmentor.comYahoo! Groups Links
                    > > >
                    > > >
                    > > >
                    > >
                    > >
                    >
                    > [Non-text portions of this message have been removed]
                    >
                    >



                    [Non-text portions of this message have been removed]
                  • JeffGrigg
                    Agitar s Agitator is an excellent tool. Probably the best one on the market. (If you want 2nd best, and also like dealing with an extremely pushy sales
                    Message 9 of 10 , Mar 20 7:57 PM
                    • 0 Attachment
                      Agitar's Agitator is an excellent tool. Probably the best one on the market.

                      (If you want 2nd best, and also like dealing with an extremely pushy sales department, look at Jtest by Parasoft too. http://www.parasoft.com/jsp/products/jtest.jsp http://en.wikipedia.org/wiki/Jtest )

                      The best such tools can do is to predict what kinds of inputs will produce odd and potentially incorrect results -- such as exceptions. They're a lot better than a sharp stick in the face, when one wants to add tests to legacy code. But as an alternative to TDD, they just don't cut it. If used in addition to TDD, I might see some value. But you still have to put in a significant amount of work to analyze, understand, and properly respond to each of the many many many "test cases" they generate.


                      --- Daniel Wildt <dwildt@...> wrote:
                      > I know this one...
                      > http://www.agitar.com/solutions/products/automated_junit_generation.html
                    Your message has been successfully submitted and would be delivered to recipients shortly.