Loading ...
Sorry, an error occurred while loading the content.

RE: [XP] Is it possible to generate code from tests?

Expand Messages
  • Tom Copeland
    I discussed this with David Dixon-Peugh (http://quilt.sf.net/) and he suggested some sort of genetic algorithm thingy. Since I don t know anything about
    Message 1 of 24 , Jul 1, 2002
    • 0 Attachment
      I discussed this with David Dixon-Peugh (http://quilt.sf.net/) and he
      suggested some sort of genetic algorithm thingy. Since I don't know
      anything about genetic algorithms, that sounds good to me :-)

      Perhaps the generator could be helped by supplying 3-4 test methods so
      that the code generator. So if it ran out of gas on one test method it
      could back off and try one of the other methods, i.e.:

      // test 1, the code generator gets at least a class definition and a
      couple method signatures from this
      public void testSize() {
      List l = new List();
      l.add(new Object());
      assertTrue(l.size() == 1);
      }

      // test 2, code generator gets a few more signatures
      public void testEmpty() {
      List l = new List();
      assertTrue(l.isEmpty());
      }

      // test 3, now the code generator has to connect add() and removeAll()
      and isEmpty() somehow... this might be tough...
      public void testEmpty() {
      List l = new List();
      l.add(new Object());
      l.removeAll();
      assertTrue(l.isEmpty());
      }

      Worth a shot though. Even if we can't get much further than this, we
      can still generate a class and a bunch of valid method signatures.

      Yours,

      Tom

      -----Original Message-----
      From: ejfried@... [mailto:ejfried@...]
      Sent: Monday, July 01, 2002 10:16 AM
      To: extremeprogramming@yahoogroups.com
      Subject: Re: [XP] Is it possible to generate code from tests?


      I think Tom Copeland wrote:
      >
      > ===================
      > 1) The test wants a List class. Hm, can't find it in the classpath,
      so
      > I'll generate a List with a no-arg constructor
      > 2) OK, the test wants a add() that takes an Object, no problem.
      > 3) OK, the test wants a size() that returns an int, cool.
      > 4) Hm, so the test failed, so it can't just be any int...
      > ===================
      >
      > etc., etc., I won't continue because I'm not sure what the generator
      > would do at that point :-)
      >

      This sounds useful, although I'm pretty sure at the point you stopped
      is where it becomes an impossible mind-reading problem, at least in
      the general case. It'd still be a nice accelerator for TDD. So,
      anybody ever tried this?


      ---------------------------------------------------------
      Ernest Friedman-Hill
      Distributed Systems Research Phone: (925) 294-2154
      Sandia National Labs FAX: (925) 294-2234
      Org. 8920, MS 9012 ejfried@...
      PO Box 969 http://herzberg.ca.sandia.gov
      Livermore, CA 94550

      To Post a message, send it to: extremeprogramming@...

      To Unsubscribe, send a blank message to:
      extremeprogramming-unsubscribe@...

      ad-free courtesy of objectmentor.com

      Your use of Yahoo! Groups is subject to
      http://docs.yahoo.com/info/terms/
    • Bryan Dollery
      Hi Tom, I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a while ago. He said that they d investigate the idea for v3 of IDEA. Cheers,
      Message 2 of 24 , Jul 1, 2002
      • 0 Attachment
        Hi Tom,

        I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a
        while ago. He said that they'd investigate the idea for v3 of IDEA.

        Cheers,

        Bryan


        > -----Original Message-----
        > From: Tom Copeland [mailto:tom@...]
        > Sent: Tuesday, July 02, 2002 2:20 AM
        > To: extremeprogramming@yahoogroups.com
        > Subject: [XP] Is it possible to generate code from tests?
        >
        >
        > I mean, if you have something like:
        >
        > ===================
        > public void testList() {
        > List l = new List();
        > l.add(new Object());
        > assertTrue(l.size() == 1);
        > l.removeAll();
        > assertTrue(l.empty());
        > }
        > ===================
        >
        > Then could some sort of mutator code generation thingy put together a
        > List class over a couple of passes like:
        >
        > ===================
        > 1) The test wants a List class. Hm, can't find it in the classpath, so
        > I'll generate a List with a no-arg constructor
        > 2) OK, the test wants a add() that takes an Object, no problem.
        > 3) OK, the test wants a size() that returns an int, cool.
        > 4) Hm, so the test failed, so it can't just be any int...
        > ===================
        >
        > etc., etc., I won't continue because I'm not sure what the generator
        > would do at that point :-)
        >
        > If this has already been discussed, sorry about that. Anybody have any
        > pointers to any projects like this? Searching Google for "junit test
        > driven code generation" didn't seem to turn up anything...
        >
        > Yours,
        >
        > Tom Copeland
        > InfoEther
        > 703-486-4543
        >
        >
        > To Post a message, send it to: extremeprogramming@...
        >
        > To Unsubscribe, send a blank message to:
        > extremeprogramming-unsubscribe@...
        >
        > ad-free courtesy of objectmentor.com
        >
        > Your use of Yahoo! Groups is subject to
        http://docs.yahoo.com/info/terms/
      • Tom Copeland
        I would expect no less from IDEAJ. It rocks. I don t see anything on their Wiki (http://www.intellij.org/twiki/bin/view/Main/WebHome) about it yet but I ll
        Message 3 of 24 , Jul 1, 2002
        • 0 Attachment
          I would expect no less from IDEAJ. It rocks. I don't see anything on
          their Wiki (http://www.intellij.org/twiki/bin/view/Main/WebHome) about
          it yet but I'll keep an eye out for it....

          Thanks,

          Tom

          -----Original Message-----
          From: Bryan Dollery [mailto:Bryan.Dollery@...]
          Sent: Monday, July 01, 2002 10:50 AM
          To: extremeprogramming@yahoogroups.com
          Subject: RE: [XP] Is it possible to generate code from tests?


          Hi Tom,

          I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a
          while ago. He said that they'd investigate the idea for v3 of IDEA.

          Cheers,

          Bryan


          > -----Original Message-----
          > From: Tom Copeland [mailto:tom@...]
          > Sent: Tuesday, July 02, 2002 2:20 AM
          > To: extremeprogramming@yahoogroups.com
          > Subject: [XP] Is it possible to generate code from tests?
          >
          >
          > I mean, if you have something like:
          >
          > ===================
          > public void testList() {
          > List l = new List();
          > l.add(new Object());
          > assertTrue(l.size() == 1);
          > l.removeAll();
          > assertTrue(l.empty());
          > }
          > ===================
          >
          > Then could some sort of mutator code generation thingy put together a
          > List class over a couple of passes like:
          >
          > ===================
          > 1) The test wants a List class. Hm, can't find it in the classpath,
          so
          > I'll generate a List with a no-arg constructor
          > 2) OK, the test wants a add() that takes an Object, no problem.
          > 3) OK, the test wants a size() that returns an int, cool.
          > 4) Hm, so the test failed, so it can't just be any int...
          > ===================
          >
          > etc., etc., I won't continue because I'm not sure what the generator
          > would do at that point :-)
          >
          > If this has already been discussed, sorry about that. Anybody have
          any
          > pointers to any projects like this? Searching Google for "junit test
          > driven code generation" didn't seem to turn up anything...
          >
          > Yours,
          >
          > Tom Copeland
          > InfoEther
          > 703-486-4543
          >
          >
          > To Post a message, send it to: extremeprogramming@...
          >
          > To Unsubscribe, send a blank message to:
          > extremeprogramming-unsubscribe@...
          >
          > ad-free courtesy of objectmentor.com
          >
          > Your use of Yahoo! Groups is subject to
          http://docs.yahoo.com/info/terms/




          To Post a message, send it to: extremeprogramming@...

          To Unsubscribe, send a blank message to:
          extremeprogramming-unsubscribe@...

          ad-free courtesy of objectmentor.com

          Your use of Yahoo! Groups is subject to
          http://docs.yahoo.com/info/terms/
        • jeffgrigg63132
          ... Actually, that s exactly what I was thinking. See: http://www.c2.com/cgi/wiki?EvolutionaryAlgorithms And realize that this approach may be slow, and won t
          Message 4 of 24 , Jul 1, 2002
          • 0 Attachment
            --- "Tom Copeland" <tom@i...> wrote:
            > I discussed this with David Dixon-Peugh (http://quilt.sf.net/)
            > and he suggested some sort of genetic algorithm thingy. Since
            > I don't know anything about genetic algorithms, that sounds
            > good to me :-)

            Actually, that's exactly what I was thinking.
            See:
            http://www.c2.com/cgi/wiki?EvolutionaryAlgorithms

            And realize that this approach may be slow, and won't necessarily
            produce the simplest code that can possibly work.
          • Tom Copeland
            Seems neat, though... do you know if there are any projects out there that have given this a whirl? Tom ... From: jeffgrigg63132 [mailto:jgrigg@mo.net] Sent:
            Message 5 of 24 , Jul 1, 2002
            • 0 Attachment
              Seems neat, though... do you know if there are any projects out there
              that have given this a whirl?

              Tom

              -----Original Message-----
              From: jeffgrigg63132 [mailto:jgrigg@...]
              Sent: Monday, July 01, 2002 11:52 AM
              To: extremeprogramming@yahoogroups.com
              Subject: [XP] Re: Is it possible to generate code from tests? --
              Evolutionary or Genetic


              --- "Tom Copeland" <tom@i...> wrote:
              > I discussed this with David Dixon-Peugh (http://quilt.sf.net/)
              > and he suggested some sort of genetic algorithm thingy. Since
              > I don't know anything about genetic algorithms, that sounds
              > good to me :-)

              Actually, that's exactly what I was thinking.
              See:
              http://www.c2.com/cgi/wiki?EvolutionaryAlgorithms

              And realize that this approach may be slow, and won't necessarily
              produce the simplest code that can possibly work.


              To Post a message, send it to: extremeprogramming@...

              To Unsubscribe, send a blank message to:
              extremeprogramming-unsubscribe@...

              ad-free courtesy of objectmentor.com

              Your use of Yahoo! Groups is subject to
              http://docs.yahoo.com/info/terms/
            • Kevin Lawrence
              ... From: Tom Copeland To: Sent: Monday, July 01, 2002 10:48 AM Subject: RE: [XP] Re: Is it possible
              Message 6 of 24 , Jul 1, 2002
              • 0 Attachment
                ----- Original Message -----
                From: "Tom Copeland" <tom@...>
                To: <extremeprogramming@yahoogroups.com>
                Sent: Monday, July 01, 2002 10:48 AM
                Subject: RE: [XP] Re: Is it possible to generate code from tests? --
                Evolutionary or Genetic


                > Seems neat, though... do you know if there are any projects out there
                > that have given this a whirl?
                >
                > Tom

                I made a start on it about a month ago with the intention of working through
                the tic-tac-toe exercise that Bill Wake had us playing with as well as the
                prime factors problem that Uncle Bob posted on the newsgroup. I got as far
                as building a genetic programming framework and finding solutions to some of
                the classic GP problems but then I started to run out of Real Life (tm) and
                put the project on hold for a while.

                A lot of people raise the issue that the solution would not be TSTTCPW - but
                I see that as a human concern not a machine concern. If the code can be
                generated by a computer, what does it matter if a human can read it ? Why
                would a human ever need to ?

                Having said that, I also have vague dreams of being able to automatically
                eliminate duplication from the evolved solution and possibly even attempt
                more ambitious refactorings automatically.

                I don't have any illusion that this will replace TDD any time soon, but I am
                curious whether it is possible - it may even be practical at some point.
                I'll report back in 15 or 20 years when I have some results.

                BTW I talked about this on a newsgroup and 3 or 4 other people emailed to
                say that they are attempting the same thing.

                Kevin
              • Laurent Bossavit
                ... And of course if you write the thing test-first it ll eventually bootstrap itself, which is when you ll know the thing works : as long as you have to
                Message 7 of 24 , Jul 1, 2002
                • 0 Attachment
                  > I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a
                  > while ago. He said that they'd investigate the idea for v3 of IDEA.

                  And of course if you write the thing test-first it'll eventually
                  bootstrap itself, which is when you'll know the thing works : as long
                  as you have to actually *think*, you're not done yet.

                  Cheers,

                  -[Morendil]-
                  In XP, we don't divide and conquer. We conquer and divide. First we
                  make something that works, then bust it up and solve the little
                  parts.
                  Kent Beck
                • Dinwiddie, George
                  Tom, If you do this, how will you test the tests? Aren t you just changing the programming language? Is it easier to specify all the behavior in the form of
                  Message 8 of 24 , Jul 1, 2002
                  • 0 Attachment
                    Tom,

                    If you do this, how will you test the tests? Aren't you just changing the
                    programming language? Is it easier to specify all the behavior in the form
                    of tests than in the programming language itself?

                    I like Test Driven Development and find that writing the tests focuses my
                    mind on what I want. I have no illusions that my tests fully specify the
                    behavior, however. There are plenty of small issues that "couldn't possibly
                    break" and have no tests.

                    To me, the verification of testing is the independent evaluation. If the
                    executable code is derived from the test, rather than derived from the
                    understanding of the problem, then it's no longer independent.

                    - George

                    > -----Original Message-----
                    > From: Tom Copeland [mailto:tom@...]
                    > Sent: Monday, July 01, 2002 10:20 AM
                    > To: extremeprogramming@yahoogroups.com
                    > Subject: [XP] Is it possible to generate code from tests?
                    >
                    >
                    > I mean, if you have something like:
                    >
                    > ===================
                    > public void testList() {
                    > List l = new List();
                    > l.add(new Object());
                    > assertTrue(l.size() == 1);
                    > l.removeAll();
                    > assertTrue(l.empty());
                    > }
                    > ===================
                    >
                    > Then could some sort of mutator code generation thingy put together a
                    > List class over a couple of passes like:
                    >
                    > ===================
                    > 1) The test wants a List class. Hm, can't find it in the
                    > classpath, so
                    > I'll generate a List with a no-arg constructor
                    > 2) OK, the test wants a add() that takes an Object, no problem.
                    > 3) OK, the test wants a size() that returns an int, cool.
                    > 4) Hm, so the test failed, so it can't just be any int...
                    > ===================
                    >
                    > etc., etc., I won't continue because I'm not sure what the generator
                    > would do at that point :-)
                    >
                    > If this has already been discussed, sorry about that.
                    > Anybody have any
                    > pointers to any projects like this? Searching Google for "junit test
                    > driven code generation" didn't seem to turn up anything...
                    >
                    > Yours,
                    >
                    > Tom Copeland
                    > InfoEther
                    > 703-486-4543
                    >
                    >
                    > To Post a message, send it to: extremeprogramming@...
                    >
                    > To Unsubscribe, send a blank message to:
                    > extremeprogramming-unsubscribe@...
                    >
                    > ad-free courtesy of objectmentor.com
                    >
                    > Your use of Yahoo! Groups is subject to
                    > http://docs.yahoo.com/info/terms/
                    >
                    >
                  • ejfried@ca.sandia.gov
                    ... I agree. I voiced some enthusiasm for this before, but only as a way to create the minimal stub class after writing a test. Write a test, compile, it
                    Message 9 of 24 , Jul 1, 2002
                    • 0 Attachment
                      I think Dinwiddie, George wrote:

                      > To me, the verification of testing is the independent evaluation. If the
                      > executable code is derived from the test, rather than derived from the
                      > understanding of the problem, then it's no longer independent.
                      >

                      I agree. I voiced some enthusiasm for this before, but only as a way
                      to create the minimal "stub class" after writing a test. Write a test,
                      compile, it won't compile. Generate a stub, compile, now it compiles
                      but fails to run. Now you start editing the generated code. This is
                      just a generalized form of the "Implement Interface" feature many Java
                      IDEs have.


                      -- Ernest Friedman-Hill
                    • jmax017
                      This is an excellent idea. But I think that it would be rather difficult to implement. I have feeling that it would have to work in a similar manner to a
                      Message 10 of 24 , Jul 1, 2002
                      • 0 Attachment
                        This is an excellent idea.
                        But I think that it would be rather difficult to implement.

                        I have feeling that it would have to work in a similar manner to a
                        Functional Programming Languages (SML,Haskell,Scheme etc.) because it
                        is another step away from command oriented lanaguages.

                        I think development of interfaces from tests would be trival (this
                        may exist already) the building of working code would be very
                        difficult.


                        Cheers,
                        Jeff Maxwell
                      • Tom Copeland
                        Hmm.... so if the code could be completely generated from the test, the test would be a formal spec for the code. That s interesting. That s true, when I m
                        Message 11 of 24 , Jul 1, 2002
                        • 0 Attachment
                          Hmm.... so if the code could be completely generated from the test, the
                          test would be a formal spec for the code. That's interesting.

                          That's true, when I'm writing tests, I'm writing them to
                          verify/exercise/use the code. That's why I don't like autogenerated
                          tests, because writing them is a good exercise for figuring out the
                          code.

                          On the other hand, if we could generate a List class from the test, we'd
                          have a pretty sweet deal - a class, a test class, and a bunch of
                          examples on how to use it. Sweet.

                          Yours,

                          Tom

                          -----Original Message-----
                          From: ejfried@... [mailto:ejfried@...]
                          Sent: Monday, July 01, 2002 2:17 PM
                          To: extremeprogramming@yahoogroups.com
                          Subject: Re: [XP] Is it possible to generate code from tests?


                          I think Dinwiddie, George wrote:

                          > To me, the verification of testing is the independent evaluation. If
                          the
                          > executable code is derived from the test, rather than derived from the
                          > understanding of the problem, then it's no longer independent.
                          >

                          I agree. I voiced some enthusiasm for this before, but only as a way
                          to create the minimal "stub class" after writing a test. Write a test,
                          compile, it won't compile. Generate a stub, compile, now it compiles
                          but fails to run. Now you start editing the generated code. This is
                          just a generalized form of the "Implement Interface" feature many Java
                          IDEs have.


                          -- Ernest Friedman-Hill

                          To Post a message, send it to: extremeprogramming@...

                          To Unsubscribe, send a blank message to:
                          extremeprogramming-unsubscribe@...

                          ad-free courtesy of objectmentor.com

                          Your use of Yahoo! Groups is subject to
                          http://docs.yahoo.com/info/terms/
                        • Tom Copeland
                          Right, just like the JavaCC generated parsers. They re gibberish, but who cares? Using something like BCEL (http://jakarta.apache.org/bcel/) would allow the
                          Message 12 of 24 , Jul 1, 2002
                          • 0 Attachment
                            Right, just like the JavaCC generated parsers. They're gibberish, but
                            who cares?

                            Using something like BCEL (http://jakarta.apache.org/bcel/) would allow
                            the "test driven code generator" to generate a test, reload it, and
                            rerun it without having to restart the VM. So it'd be pretty zippy in
                            that regard. Of course, it might still take a million years or so to do
                            :-)

                            Funky stuff!

                            Tom

                            -----Original Message-----
                            From: Kevin Lawrence [mailto:kevin@...]
                            Sent: Monday, July 01, 2002 1:31 PM
                            To: extremeprogramming@yahoogroups.com
                            Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                            Evolutionary or Genetic


                            I made a start on it about a month ago with the intention of working
                            through
                            the tic-tac-toe exercise that Bill Wake had us playing with as well as
                            the
                            prime factors problem that Uncle Bob posted on the newsgroup. I got as
                            far
                            as building a genetic programming framework and finding solutions to
                            some of
                            the classic GP problems but then I started to run out of Real Life (tm)
                            and
                            put the project on hold for a while.

                            > A lot of people raise the issue that the solution would not be TSTTCPW
                            - but
                            > I see that as a human concern not a machine concern. If the code can
                            be
                            > generated by a computer, what does it matter if a human can read it ?
                            Why
                            > would a human ever need to ?

                            Having said that, I also have vague dreams of being able to
                            automatically
                            eliminate duplication from the evolved solution and possibly even
                            attempt
                            more ambitious refactorings automatically.

                            I don't have any illusion that this will replace TDD any time soon, but
                            I am
                            curious whether it is possible - it may even be practical at some point.
                            I'll report back in 15 or 20 years when I have some results.

                            BTW I talked about this on a newsgroup and 3 or 4 other people emailed
                            to
                            say that they are attempting the same thing.

                            Kevin




                            To Post a message, send it to: extremeprogramming@...

                            To Unsubscribe, send a blank message to:
                            extremeprogramming-unsubscribe@...

                            ad-free courtesy of objectmentor.com

                            Your use of Yahoo! Groups is subject to
                            http://docs.yahoo.com/info/terms/
                          • Richard Faust
                            Constraint Programming, maybe? Constraint programming represents one of the closest approaches computer science has yet made to the Holy Grail of programming:
                            Message 13 of 24 , Jul 1, 2002
                            • 0 Attachment
                              Constraint Programming, maybe?

                              "Constraint programming represents one of the closest approaches computer
                              science has yet made to the Holy Grail of programming: the user states the
                              problem, the computer solves it."
                              Eugene C. Freuder, April 1997
                              http://www.kluweronline.com/issn/1383-7133

                              At 19:32 1/7/2002 +0000, you wrote:
                              >This is an excellent idea.
                              >But I think that it would be rather difficult to implement.
                              >
                              >I have feeling that it would have to work in a similar manner to a
                              >Functional Programming Languages (SML,Haskell,Scheme etc.) because it
                              >is another step away from command oriented lanaguages.
                              >
                              >I think development of interfaces from tests would be trival (this
                              >may exist already) the building of working code would be very
                              >difficult.
                              >
                              >
                              >Cheers,
                              >Jeff Maxwell
                              >
                              >
                              >
                              >
                              >
                              >
                              >
                              >
                              >To Post a message, send it to: extremeprogramming@...
                              >
                              >To Unsubscribe, send a blank message to:
                              >extremeprogramming-unsubscribe@...
                              >
                              >ad-free courtesy of objectmentor.com
                              >
                              >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                            • jmax017
                              Yes I imagine that would be similar. Test are just a method of stating the problem. Cheers, Jeff Maxwell ... computer ... states the ... it ...
                              Message 14 of 24 , Jul 1, 2002
                              • 0 Attachment
                                Yes I imagine that would be similar.
                                Test are just a method of stating the problem.

                                Cheers,
                                Jeff Maxwell

                                --- In extremeprogramming@y..., Richard Faust <richard.faust@i...>
                                wrote:
                                > Constraint Programming, maybe?
                                >
                                > "Constraint programming represents one of the closest approaches
                                computer
                                > science has yet made to the Holy Grail of programming: the user
                                states the
                                > problem, the computer solves it."
                                > Eugene C. Freuder, April 1997
                                > http://www.kluweronline.com/issn/1383-7133
                                >
                                > At 19:32 1/7/2002 +0000, you wrote:
                                > >This is an excellent idea.
                                > >But I think that it would be rather difficult to implement.
                                > >
                                > >I have feeling that it would have to work in a similar manner to a
                                > >Functional Programming Languages (SML,Haskell,Scheme etc.) because
                                it
                                > >is another step away from command oriented lanaguages.
                                > >
                                > >I think development of interfaces from tests would be trival (this
                                > >may exist already) the building of working code would be very
                                > >difficult.
                                > >
                                > >
                                > >Cheers,
                                > >Jeff Maxwell
                                > >
                                > >
                                > >
                                > >
                                > >
                                > >
                                > >
                                > >
                                > >To Post a message, send it to: extremeprogramming@e...
                                > >
                                > >To Unsubscribe, send a blank message to:
                                > >extremeprogramming-unsubscribe@e...
                                > >
                                > >ad-free courtesy of objectmentor.com
                                > >
                                > >Your use of Yahoo! Groups is subject to
                                http://docs.yahoo.com/info/terms/
                              • jmax017
                                Correct something Like javaCC. JavaCC has a very nice advantage of using grammers to build its parsers and grammers are nice and mathametical. Can your test be
                                Message 15 of 24 , Jul 1, 2002
                                • 0 Attachment
                                  Correct something Like javaCC.

                                  JavaCC has a very nice advantage of using grammers to build its
                                  parsers and grammers are nice and mathametical.

                                  Can your test be as mathamatical?

                                  Cheers,
                                  Jeff Maxwell



                                  --- In extremeprogramming@y..., "Tom Copeland" <tom@i...> wrote:
                                  > Right, just like the JavaCC generated parsers. They're gibberish,
                                  but
                                  > who cares?
                                  >
                                  > Using something like BCEL (http://jakarta.apache.org/bcel/) would
                                  allow
                                  > the "test driven code generator" to generate a test, reload it, and
                                  > rerun it without having to restart the VM. So it'd be pretty zippy
                                  in
                                  > that regard. Of course, it might still take a million years or so
                                  to do
                                  > :-)
                                  >
                                  > Funky stuff!
                                  >
                                  > Tom
                                  >
                                  > -----Original Message-----
                                  > From: Kevin Lawrence [mailto:kevin@d...]
                                  > Sent: Monday, July 01, 2002 1:31 PM
                                  > To: extremeprogramming@y...
                                  > Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                                  > Evolutionary or Genetic
                                  >
                                  >
                                  > I made a start on it about a month ago with the intention of working
                                  > through
                                  > the tic-tac-toe exercise that Bill Wake had us playing with as well
                                  as
                                  > the
                                  > prime factors problem that Uncle Bob posted on the newsgroup. I got
                                  as
                                  > far
                                  > as building a genetic programming framework and finding solutions to
                                  > some of
                                  > the classic GP problems but then I started to run out of Real Life
                                  (tm)
                                  > and
                                  > put the project on hold for a while.
                                  >
                                  > > A lot of people raise the issue that the solution would not be
                                  TSTTCPW
                                  > - but
                                  > > I see that as a human concern not a machine concern. If the code
                                  can
                                  > be
                                  > > generated by a computer, what does it matter if a human can read
                                  it ?
                                  > Why
                                  > > would a human ever need to ?
                                  >
                                  > Having said that, I also have vague dreams of being able to
                                  > automatically
                                  > eliminate duplication from the evolved solution and possibly even
                                  > attempt
                                  > more ambitious refactorings automatically.
                                  >
                                  > I don't have any illusion that this will replace TDD any time soon,
                                  but
                                  > I am
                                  > curious whether it is possible - it may even be practical at some
                                  point.
                                  > I'll report back in 15 or 20 years when I have some results.
                                  >
                                  > BTW I talked about this on a newsgroup and 3 or 4 other people
                                  emailed
                                  > to
                                  > say that they are attempting the same thing.
                                  >
                                  > Kevin
                                  >
                                  >
                                  >
                                  >
                                  > To Post a message, send it to: extremeprogramming@e...
                                  >
                                  > To Unsubscribe, send a blank message to:
                                  > extremeprogramming-unsubscribe@e...
                                  >
                                  > ad-free courtesy of objectmentor.com
                                  >
                                  > Your use of Yahoo! Groups is subject to
                                  > http://docs.yahoo.com/info/terms/
                                • Tom Copeland
                                  Yup, true, my unit tests usually aren t nearly as precise as an EBNF grammer. Still though, in some cases I ve ended up with a suite of tests that specified a
                                  Message 16 of 24 , Jul 1, 2002
                                  • 0 Attachment
                                    Yup, true, my unit tests usually aren't nearly as precise as an EBNF
                                    grammer.

                                    Still though, in some cases I've ended up with a suite of tests that
                                    specified a class' behavior with pretty good preciseness - at least, I
                                    think it did. I dunno.... I kind of feel like tests would overlap in
                                    ways that would cover a lot of ground... and an implementation that
                                    passed all the tests would be pretty good.

                                    Of course, that gets us back to: even if the tests are quite
                                    comprehensive, how does the code generator put them together to make a
                                    real Java class? And thence into the depths of all sorts of genetic
                                    programming niftiness. Cool stuff to think about though....

                                    Yours,

                                    Tom

                                    -----Original Message-----
                                    From: jmax017 [mailto:jmax01@...]
                                    Sent: Monday, July 01, 2002 4:30 PM
                                    To: extremeprogramming@yahoogroups.com
                                    Subject: [XP] Re: Is it possible to generate code from tests? --
                                    Evolutionary or Genetic


                                    Correct something Like javaCC.

                                    JavaCC has a very nice advantage of using grammers to build its
                                    parsers and grammers are nice and mathametical.

                                    Can your test be as mathamatical?

                                    Cheers,
                                    Jeff Maxwell

                                    <snip>
                                  • hpforever
                                    IDEA can generate method stub from its usage. E.g. from the Tom test case below following class can be generated automagically: public class List { public
                                    Message 17 of 24 , Jul 2, 2002
                                    • 0 Attachment
                                      IDEA can generate method stub from its usage.
                                      E.g. from the Tom' test case below following class can be generated
                                      automagically:
                                      public class List {
                                      public void add(Object o) {}
                                      public void removeAll() {}
                                      public int size() { return 0; }
                                      public boolean empty() { return false; }
                                      }
                                      As for realizing add() and size() relation and figuring out
                                      corresponding algorithms, I afraid, it is Universal Problem Solver
                                      job.
                                      http://www.personal.leeds.ac.uk/~ecl6nb/OnTarget/1989/war/war92cov.ht
                                      m

                                      --
                                      regards,
                                      Alexey Kudravtsev
                                      cdrATintellij.com

                                      --- In extremeprogramming@y..., "Bryan Dollery" <Bryan.Dollery@C...>
                                      wrote:
                                      > Hi Tom,
                                      >
                                      > I suggested the same thing to Eugene Balyaev, the manager of
                                      IntelliJ a
                                      > while ago. He said that they'd investigate the idea for v3 of IDEA.
                                      >
                                      > Cheers,
                                      >
                                      > Bryan
                                      >
                                      >
                                      > > -----Original Message-----
                                      > > From: Tom Copeland [mailto:tom@i...]
                                      > > Sent: Tuesday, July 02, 2002 2:20 AM
                                      > > To: extremeprogramming@y...
                                      > > Subject: [XP] Is it possible to generate code from tests?
                                      > >
                                      > >
                                      > > I mean, if you have something like:
                                      > >
                                      > > ===================
                                      > > public void testList() {
                                      > > List l = new List();
                                      > > l.add(new Object());
                                      > > assertTrue(l.size() == 1);
                                      > > l.removeAll();
                                      > > assertTrue(l.empty());
                                      > > }
                                      > > ===================
                                      > >
                                      > > Then could some sort of mutator code generation thingy put
                                      together a
                                      > > List class over a couple of passes like:
                                      > >
                                      > > ===================
                                      > > 1) The test wants a List class. Hm, can't find it in the
                                      classpath, so
                                      > > I'll generate a List with a no-arg constructor
                                      > > 2) OK, the test wants a add() that takes an Object, no problem.
                                      > > 3) OK, the test wants a size() that returns an int, cool.
                                      > > 4) Hm, so the test failed, so it can't just be any int...
                                      > > ===================
                                      > >
                                      > > etc., etc., I won't continue because I'm not sure what the
                                      generator
                                      > > would do at that point :-)
                                      > >
                                      > > If this has already been discussed, sorry about that. Anybody
                                      have any
                                      > > pointers to any projects like this? Searching Google for "junit
                                      test
                                      > > driven code generation" didn't seem to turn up anything...
                                      > >
                                    • Robert Blum
                                      Hi Tom, ... Would you still think that if you renamed all the variables and methods to foo(), bar, and similar names? Serious question. At least for me, a good
                                      Message 18 of 24 , Jul 2, 2002
                                      • 0 Attachment
                                        Hi Tom,

                                        >Still though, in some cases I've ended up with a suite of tests that
                                        >specified a class' behavior with pretty good preciseness - at least, I
                                        >think it did.
                                        >
                                        Would you still think that if you renamed all the variables and methods
                                        to foo(), bar, and similar
                                        names? Serious question. At least for me, a good part of the value of a
                                        test is in its readability.
                                        I /know/ what add() and size() (should) do - the code generator might
                                        have a problem.

                                        >I dunno.... I kind of feel like tests would overlap in
                                        >ways that would cover a lot of ground... and an implementation that
                                        >passed all the tests would be pretty good.
                                        >
                                        One problem would be to determine if pretty good is sufficiently good.
                                        As we've seen in the
                                        'binary search' thread, it's quite easy to create an algorithm that
                                        works most of the time. :)

                                        We can then use tests to explore the areas where we are not sure if the
                                        code is behaving
                                        properly: boundary conditions, for example. With automatically generated
                                        code, that might
                                        be more difficult to do, simply because we might not be able to quickly
                                        understand what
                                        the code does.

                                        Nevertheless, it sounds like a very interesting thing to have. I wonder
                                        if there's any research
                                        on the topic....


                                        - Robert
                                      • Tom Copeland
                                        Yeah, the variable name thing is something. Here s an excerpt fm JavaParser (generated by JavaCC): =========== if (jj_2_30(2147483647)) {
                                        Message 19 of 24 , Jul 2, 2002
                                        • 0 Attachment
                                          Yeah, the variable name thing is something. Here's an excerpt fm
                                          JavaParser (generated by JavaCC):

                                          ===========
                                          if (jj_2_30(2147483647)) {
                                          LocalVariableDeclaration();
                                          } else {
                                          switch ((jj_ntk==-1)?jj_ntk():jj_ntk) {
                                          ===========

                                          Yikes. On the other hand, it's not meant to be read, and the parts that
                                          are meant to be read - i.e., the grammer - are much clearer.

                                          If the generated code passed the tests, though, all the externally
                                          visible symbols - function names and whatnot - would all have reasonable
                                          names, because they'd be specified by the test. Internally it might be
                                          gibberish... but since that would be private and local variables, maybe
                                          that's not a big deal?

                                          My wife is a mathematician and, when I talked to her about this,
                                          immediately suggested recurrence relations. So the computer is given
                                          "1,3,5,7" and has to come up with "2n+1" as the solution. I'm not sure
                                          how that would translate to code, though... there are a lot more
                                          operators and operands to work with.

                                          Maybe it's some sort of neural net thingy, where you seed the net with a
                                          bunch of sample patterns - i.e., a put() followed by a get() implies an
                                          instance variable to maintain state. Maybe you could feed the JDK
                                          source into the neural net and that would be the learning mechanism.
                                          Maybe that sounds reasonable to me just because I know nothing
                                          whatsoever about neural nets :-)

                                          Yours,

                                          Tom

                                          -----Original Message-----
                                          From: Robert Blum [mailto:r.blum@...]
                                          Sent: Tuesday, July 02, 2002 9:18 AM
                                          To: extremeprogramming@yahoogroups.com
                                          Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                                          Evolutionary or Genetic


                                          Hi Tom,

                                          >Still though, in some cases I've ended up with a suite of tests that
                                          >specified a class' behavior with pretty good preciseness - at least, I
                                          >think it did.
                                          >
                                          Would you still think that if you renamed all the variables and methods
                                          to foo(), bar, and similar
                                          names? Serious question. At least for me, a good part of the value of a
                                          test is in its readability.
                                          I /know/ what add() and size() (should) do - the code generator might
                                          have a problem.

                                          >I dunno.... I kind of feel like tests would overlap in
                                          >ways that would cover a lot of ground... and an implementation that
                                          >passed all the tests would be pretty good.
                                          >
                                          One problem would be to determine if pretty good is sufficiently good.
                                          As we've seen in the
                                          'binary search' thread, it's quite easy to create an algorithm that
                                          works most of the time. :)

                                          We can then use tests to explore the areas where we are not sure if the
                                          code is behaving
                                          properly: boundary conditions, for example. With automatically generated

                                          code, that might
                                          be more difficult to do, simply because we might not be able to quickly
                                          understand what
                                          the code does.

                                          Nevertheless, it sounds like a very interesting thing to have. I wonder
                                          if there's any research
                                          on the topic....


                                          - Robert


                                          To Post a message, send it to: extremeprogramming@...

                                          To Unsubscribe, send a blank message to:
                                          extremeprogramming-unsubscribe@...

                                          ad-free courtesy of objectmentor.com

                                          Your use of Yahoo! Groups is subject to
                                          http://docs.yahoo.com/info/terms/
                                        • Griffin Caprio
                                          I think it would be possible to at least generate the signatures. After all, many UML tools can reverse engineer/generate code. Griffin ... Griffin Caprio
                                          Message 20 of 24 , Jul 2, 2002
                                          • 0 Attachment
                                            I think it would be possible to at least generate the signatures. After
                                            all, many UML tools can reverse engineer/generate code.

                                            Griffin


                                            >From: ejfried@...
                                            >Reply-To: extremeprogramming@yahoogroups.com
                                            >To: extremeprogramming@yahoogroups.com
                                            >Subject: Re: [XP] Is it possible to generate code from tests?
                                            >Date: Mon, 1 Jul 2002 08:15:47 -0700 (PDT)
                                            >
                                            >I think Tom Copeland wrote:
                                            > >
                                            > > ===================
                                            > > 1) The test wants a List class. Hm, can't find it in the classpath, so
                                            > > I'll generate a List with a no-arg constructor
                                            > > 2) OK, the test wants a add() that takes an Object, no problem.
                                            > > 3) OK, the test wants a size() that returns an int, cool.
                                            > > 4) Hm, so the test failed, so it can't just be any int...
                                            > > ===================
                                            > >
                                            > > etc., etc., I won't continue because I'm not sure what the generator
                                            > > would do at that point :-)
                                            > >
                                            >
                                            >This sounds useful, although I'm pretty sure at the point you stopped
                                            >is where it becomes an impossible mind-reading problem, at least in
                                            >the general case. It'd still be a nice accelerator for TDD. So,
                                            >anybody ever tried this?
                                            >
                                            >
                                            >---------------------------------------------------------
                                            >Ernest Friedman-Hill
                                            >Distributed Systems Research Phone: (925) 294-2154
                                            >Sandia National Labs FAX: (925) 294-2234
                                            >Org. 8920, MS 9012 ejfried@...
                                            >PO Box 969 http://herzberg.ca.sandia.gov
                                            >Livermore, CA 94550
                                            >
                                            >To Post a message, send it to: extremeprogramming@...
                                            >
                                            >To Unsubscribe, send a blank message to:
                                            >extremeprogramming-unsubscribe@...
                                            >
                                            >ad-free courtesy of objectmentor.com
                                            >
                                            >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/




                                            Griffin Caprio
                                            cell: (773) 230-0936


                                            _________________________________________________________________
                                            Send and receive Hotmail on your mobile device: http://mobile.msn.com
                                          • Robert Blum
                                            Hi Tom, ... [snip] ... Ah, I was not talking about the generated code. Most people tend to just live with (compiler-) generated code and never look at them.
                                            Message 21 of 24 , Jul 6, 2002
                                            • 0 Attachment
                                              Hi Tom,

                                              >Yeah, the variable name thing is something. Here's an excerpt fm
                                              >JavaParser (generated by JavaCC):
                                              >
                                              [snip]

                                              >Yikes. On the other hand, it's not meant to be read, and the parts that
                                              >are meant to be read - i.e., the grammer - are much clearer.

                                              Ah, I was not talking about the generated code. Most people tend to just
                                              live with
                                              (compiler-) generated code and never look at them. But the test cases
                                              carry extra
                                              information in the names of the functions and variables they use that is
                                              inaccessible
                                              to the automated code generator.

                                              Unless the code generator is also able to understand human language.
                                              E.g. if I write
                                              a test for a function called EstimateDmaTransferSize, I know just by
                                              looking at the
                                              name what the function is supposed to do.

                                              But this requires knowledge about what a DMA transfer is, and how you
                                              estimate
                                              its size for the platform in question. Nothing an automated code
                                              generator in the
                                              near future will have.

                                              >
                                              >gibberish... but since that would be private and local variables, maybe
                                              >that's not a big deal?
                                              >
                                              It is not a big deal. The big deal is in the information contained in
                                              the public name, yet
                                              not accessible to executable code.

                                              >My wife is a mathematician and, when I talked to her about this,
                                              >immediately suggested recurrence relations. So the computer is given
                                              >"1,3,5,7" and has to come up with "2n+1" as the solution.
                                              >
                                              Yes, but what if my function is named GenerateAllPrimesIncludingOne? Of
                                              course,
                                              you can argue that in this case my test case is not complete. But as
                                              long as you do
                                              not cover a functions output exhaustively, there will always be a
                                              function that does
                                              /almost/ the same thing as what you intended.

                                              Unit Tests are never complete. They're a safety net for our
                                              refactorings, a design
                                              tool, and a cure for our fear of breaking things. But they are not a
                                              complete
                                              specification of the system. Not even if you include ATs.

                                              >Maybe it's some sort of neural net thingy, where you seed the net with a
                                              >bunch of sample patterns - i.e., a put() followed by a get() implies an
                                              >instance variable to maintain state. Maybe you could feed the JDK
                                              >source into the neural net and that would be the learning mechanism.
                                              >Maybe that sounds reasonable to me just because I know nothing
                                              >whatsoever about neural nets :-)
                                              >
                                              Since I have only limited knowledge of NNs myself, take the following
                                              with a
                                              couple of salt shakers: (And I'm sure there's somebody here on the list
                                              who can
                                              provide enlightnment for us)

                                              AFAIK, a NN is good at pattern recognition. That means that a neural
                                              net could
                                              help you recognize that you're trying to build something that has been
                                              built before.

                                              So it might ( or might not) provide some guidance - but unless all
                                              programs follow
                                              the same set of patterns (some of them hidden from view), there's not
                                              much to be
                                              gained. NNs seem to be rather bad at coming up with new, inventive
                                              solutions.

                                              And their main drawback is that they don't always do what you expect
                                              them to do.
                                              Bad training data has bad results. Your above NN might have a tendency
                                              to re-
                                              write the JDK time and again. (Hey, is maybe SUN using a NN? :)

                                              - Robert
                                            • Tom Copeland
                                              Yup, after scanning a bunch of genetic programming stuff at the local MicroCenter (Lee Highway, northern VA, has an excellent technical library), I have to
                                              Message 22 of 24 , Jul 8, 2002
                                              • 0 Attachment
                                                Yup, after scanning a bunch of genetic programming stuff at the local
                                                MicroCenter (Lee Highway, northern VA, has an excellent technical
                                                library), I have to agree. Generating an implementation from unit tests
                                                seems to 1) be impossible and 2) be worthless even if it was possible.
                                                Ah well.... fun to think about anyhow...

                                                Yours,

                                                Tom

                                                -----Original Message-----
                                                From: Robert Blum [mailto:r.blum@...]
                                                Sent: Saturday, July 06, 2002 1:44 PM
                                                To: extremeprogramming@yahoogroups.com
                                                Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                                                Evolutionary or Genetic


                                                Hi Tom,

                                                >Yeah, the variable name thing is something. Here's an excerpt fm
                                                >JavaParser (generated by JavaCC):
                                                >
                                                [snip]

                                                >Yikes. On the other hand, it's not meant to be read, and the parts
                                                that
                                                >are meant to be read - i.e., the grammer - are much clearer.

                                                Ah, I was not talking about the generated code. Most people tend to just

                                                live with
                                                (compiler-) generated code and never look at them. But the test cases
                                                carry extra
                                                information in the names of the functions and variables they use that is

                                                inaccessible
                                                to the automated code generator.

                                                Unless the code generator is also able to understand human language.
                                                E.g. if I write
                                                a test for a function called EstimateDmaTransferSize, I know just by
                                                looking at the
                                                name what the function is supposed to do.

                                                But this requires knowledge about what a DMA transfer is, and how you
                                                estimate
                                                its size for the platform in question. Nothing an automated code
                                                generator in the
                                                near future will have.

                                                >
                                                >gibberish... but since that would be private and local variables, maybe
                                                >that's not a big deal?
                                                >
                                                It is not a big deal. The big deal is in the information contained in
                                                the public name, yet
                                                not accessible to executable code.

                                                >My wife is a mathematician and, when I talked to her about this,
                                                >immediately suggested recurrence relations. So the computer is given
                                                >"1,3,5,7" and has to come up with "2n+1" as the solution.
                                                >
                                                Yes, but what if my function is named GenerateAllPrimesIncludingOne? Of

                                                course,
                                                you can argue that in this case my test case is not complete. But as
                                                long as you do
                                                not cover a functions output exhaustively, there will always be a
                                                function that does
                                                /almost/ the same thing as what you intended.

                                                Unit Tests are never complete. They're a safety net for our
                                                refactorings, a design
                                                tool, and a cure for our fear of breaking things. But they are not a
                                                complete
                                                specification of the system. Not even if you include ATs.

                                                >Maybe it's some sort of neural net thingy, where you seed the net with
                                                a
                                                >bunch of sample patterns - i.e., a put() followed by a get() implies an
                                                >instance variable to maintain state. Maybe you could feed the JDK
                                                >source into the neural net and that would be the learning mechanism.
                                                >Maybe that sounds reasonable to me just because I know nothing
                                                >whatsoever about neural nets :-)
                                                >
                                                Since I have only limited knowledge of NNs myself, take the following
                                                with a
                                                couple of salt shakers: (And I'm sure there's somebody here on the list
                                                who can
                                                provide enlightnment for us)

                                                AFAIK, a NN is good at pattern recognition. That means that a neural
                                                net could
                                                help you recognize that you're trying to build something that has been
                                                built before.

                                                So it might ( or might not) provide some guidance - but unless all
                                                programs follow
                                                the same set of patterns (some of them hidden from view), there's not
                                                much to be
                                                gained. NNs seem to be rather bad at coming up with new, inventive
                                                solutions.

                                                And their main drawback is that they don't always do what you expect
                                                them to do.
                                                Bad training data has bad results. Your above NN might have a tendency
                                                to re-
                                                write the JDK time and again. (Hey, is maybe SUN using a NN? :)

                                                - Robert


                                                To Post a message, send it to: extremeprogramming@...

                                                To Unsubscribe, send a blank message to:
                                                extremeprogramming-unsubscribe@...

                                                ad-free courtesy of objectmentor.com

                                                Your use of Yahoo! Groups is subject to
                                                http://docs.yahoo.com/info/terms/
                                              Your message has been successfully submitted and would be delivered to recipients shortly.