Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Is it possible to generate code from tests?

Expand Messages
  • ejfried@ca.sandia.gov
    ... This sounds useful, although I m pretty sure at the point you stopped is where it becomes an impossible mind-reading problem, at least in the general case.
    Message 1 of 24 , Jul 1 8:15 AM
    • 0 Attachment
      I think Tom Copeland wrote:
      >
      > ===================
      > 1) The test wants a List class. Hm, can't find it in the classpath, so
      > I'll generate a List with a no-arg constructor
      > 2) OK, the test wants a add() that takes an Object, no problem.
      > 3) OK, the test wants a size() that returns an int, cool.
      > 4) Hm, so the test failed, so it can't just be any int...
      > ===================
      >
      > etc., etc., I won't continue because I'm not sure what the generator
      > would do at that point :-)
      >

      This sounds useful, although I'm pretty sure at the point you stopped
      is where it becomes an impossible mind-reading problem, at least in
      the general case. It'd still be a nice accelerator for TDD. So,
      anybody ever tried this?


      ---------------------------------------------------------
      Ernest Friedman-Hill
      Distributed Systems Research Phone: (925) 294-2154
      Sandia National Labs FAX: (925) 294-2234
      Org. 8920, MS 9012 ejfried@...
      PO Box 969 http://herzberg.ca.sandia.gov
      Livermore, CA 94550
    • Tom Copeland
      I discussed this with David Dixon-Peugh (http://quilt.sf.net/) and he suggested some sort of genetic algorithm thingy. Since I don t know anything about
      Message 2 of 24 , Jul 1 8:32 AM
      • 0 Attachment
        I discussed this with David Dixon-Peugh (http://quilt.sf.net/) and he
        suggested some sort of genetic algorithm thingy. Since I don't know
        anything about genetic algorithms, that sounds good to me :-)

        Perhaps the generator could be helped by supplying 3-4 test methods so
        that the code generator. So if it ran out of gas on one test method it
        could back off and try one of the other methods, i.e.:

        // test 1, the code generator gets at least a class definition and a
        couple method signatures from this
        public void testSize() {
        List l = new List();
        l.add(new Object());
        assertTrue(l.size() == 1);
        }

        // test 2, code generator gets a few more signatures
        public void testEmpty() {
        List l = new List();
        assertTrue(l.isEmpty());
        }

        // test 3, now the code generator has to connect add() and removeAll()
        and isEmpty() somehow... this might be tough...
        public void testEmpty() {
        List l = new List();
        l.add(new Object());
        l.removeAll();
        assertTrue(l.isEmpty());
        }

        Worth a shot though. Even if we can't get much further than this, we
        can still generate a class and a bunch of valid method signatures.

        Yours,

        Tom

        -----Original Message-----
        From: ejfried@... [mailto:ejfried@...]
        Sent: Monday, July 01, 2002 10:16 AM
        To: extremeprogramming@yahoogroups.com
        Subject: Re: [XP] Is it possible to generate code from tests?


        I think Tom Copeland wrote:
        >
        > ===================
        > 1) The test wants a List class. Hm, can't find it in the classpath,
        so
        > I'll generate a List with a no-arg constructor
        > 2) OK, the test wants a add() that takes an Object, no problem.
        > 3) OK, the test wants a size() that returns an int, cool.
        > 4) Hm, so the test failed, so it can't just be any int...
        > ===================
        >
        > etc., etc., I won't continue because I'm not sure what the generator
        > would do at that point :-)
        >

        This sounds useful, although I'm pretty sure at the point you stopped
        is where it becomes an impossible mind-reading problem, at least in
        the general case. It'd still be a nice accelerator for TDD. So,
        anybody ever tried this?


        ---------------------------------------------------------
        Ernest Friedman-Hill
        Distributed Systems Research Phone: (925) 294-2154
        Sandia National Labs FAX: (925) 294-2234
        Org. 8920, MS 9012 ejfried@...
        PO Box 969 http://herzberg.ca.sandia.gov
        Livermore, CA 94550

        To Post a message, send it to: extremeprogramming@...

        To Unsubscribe, send a blank message to:
        extremeprogramming-unsubscribe@...

        ad-free courtesy of objectmentor.com

        Your use of Yahoo! Groups is subject to
        http://docs.yahoo.com/info/terms/
      • Bryan Dollery
        Hi Tom, I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a while ago. He said that they d investigate the idea for v3 of IDEA. Cheers,
        Message 3 of 24 , Jul 1 8:49 AM
        • 0 Attachment
          Hi Tom,

          I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a
          while ago. He said that they'd investigate the idea for v3 of IDEA.

          Cheers,

          Bryan


          > -----Original Message-----
          > From: Tom Copeland [mailto:tom@...]
          > Sent: Tuesday, July 02, 2002 2:20 AM
          > To: extremeprogramming@yahoogroups.com
          > Subject: [XP] Is it possible to generate code from tests?
          >
          >
          > I mean, if you have something like:
          >
          > ===================
          > public void testList() {
          > List l = new List();
          > l.add(new Object());
          > assertTrue(l.size() == 1);
          > l.removeAll();
          > assertTrue(l.empty());
          > }
          > ===================
          >
          > Then could some sort of mutator code generation thingy put together a
          > List class over a couple of passes like:
          >
          > ===================
          > 1) The test wants a List class. Hm, can't find it in the classpath, so
          > I'll generate a List with a no-arg constructor
          > 2) OK, the test wants a add() that takes an Object, no problem.
          > 3) OK, the test wants a size() that returns an int, cool.
          > 4) Hm, so the test failed, so it can't just be any int...
          > ===================
          >
          > etc., etc., I won't continue because I'm not sure what the generator
          > would do at that point :-)
          >
          > If this has already been discussed, sorry about that. Anybody have any
          > pointers to any projects like this? Searching Google for "junit test
          > driven code generation" didn't seem to turn up anything...
          >
          > Yours,
          >
          > Tom Copeland
          > InfoEther
          > 703-486-4543
          >
          >
          > To Post a message, send it to: extremeprogramming@...
          >
          > To Unsubscribe, send a blank message to:
          > extremeprogramming-unsubscribe@...
          >
          > ad-free courtesy of objectmentor.com
          >
          > Your use of Yahoo! Groups is subject to
          http://docs.yahoo.com/info/terms/
        • Tom Copeland
          I would expect no less from IDEAJ. It rocks. I don t see anything on their Wiki (http://www.intellij.org/twiki/bin/view/Main/WebHome) about it yet but I ll
          Message 4 of 24 , Jul 1 9:00 AM
          • 0 Attachment
            I would expect no less from IDEAJ. It rocks. I don't see anything on
            their Wiki (http://www.intellij.org/twiki/bin/view/Main/WebHome) about
            it yet but I'll keep an eye out for it....

            Thanks,

            Tom

            -----Original Message-----
            From: Bryan Dollery [mailto:Bryan.Dollery@...]
            Sent: Monday, July 01, 2002 10:50 AM
            To: extremeprogramming@yahoogroups.com
            Subject: RE: [XP] Is it possible to generate code from tests?


            Hi Tom,

            I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a
            while ago. He said that they'd investigate the idea for v3 of IDEA.

            Cheers,

            Bryan


            > -----Original Message-----
            > From: Tom Copeland [mailto:tom@...]
            > Sent: Tuesday, July 02, 2002 2:20 AM
            > To: extremeprogramming@yahoogroups.com
            > Subject: [XP] Is it possible to generate code from tests?
            >
            >
            > I mean, if you have something like:
            >
            > ===================
            > public void testList() {
            > List l = new List();
            > l.add(new Object());
            > assertTrue(l.size() == 1);
            > l.removeAll();
            > assertTrue(l.empty());
            > }
            > ===================
            >
            > Then could some sort of mutator code generation thingy put together a
            > List class over a couple of passes like:
            >
            > ===================
            > 1) The test wants a List class. Hm, can't find it in the classpath,
            so
            > I'll generate a List with a no-arg constructor
            > 2) OK, the test wants a add() that takes an Object, no problem.
            > 3) OK, the test wants a size() that returns an int, cool.
            > 4) Hm, so the test failed, so it can't just be any int...
            > ===================
            >
            > etc., etc., I won't continue because I'm not sure what the generator
            > would do at that point :-)
            >
            > If this has already been discussed, sorry about that. Anybody have
            any
            > pointers to any projects like this? Searching Google for "junit test
            > driven code generation" didn't seem to turn up anything...
            >
            > Yours,
            >
            > Tom Copeland
            > InfoEther
            > 703-486-4543
            >
            >
            > To Post a message, send it to: extremeprogramming@...
            >
            > To Unsubscribe, send a blank message to:
            > extremeprogramming-unsubscribe@...
            >
            > ad-free courtesy of objectmentor.com
            >
            > Your use of Yahoo! Groups is subject to
            http://docs.yahoo.com/info/terms/




            To Post a message, send it to: extremeprogramming@...

            To Unsubscribe, send a blank message to:
            extremeprogramming-unsubscribe@...

            ad-free courtesy of objectmentor.com

            Your use of Yahoo! Groups is subject to
            http://docs.yahoo.com/info/terms/
          • jeffgrigg63132
            ... Actually, that s exactly what I was thinking. See: http://www.c2.com/cgi/wiki?EvolutionaryAlgorithms And realize that this approach may be slow, and won t
            Message 5 of 24 , Jul 1 9:51 AM
            • 0 Attachment
              --- "Tom Copeland" <tom@i...> wrote:
              > I discussed this with David Dixon-Peugh (http://quilt.sf.net/)
              > and he suggested some sort of genetic algorithm thingy. Since
              > I don't know anything about genetic algorithms, that sounds
              > good to me :-)

              Actually, that's exactly what I was thinking.
              See:
              http://www.c2.com/cgi/wiki?EvolutionaryAlgorithms

              And realize that this approach may be slow, and won't necessarily
              produce the simplest code that can possibly work.
            • Tom Copeland
              Seems neat, though... do you know if there are any projects out there that have given this a whirl? Tom ... From: jeffgrigg63132 [mailto:jgrigg@mo.net] Sent:
              Message 6 of 24 , Jul 1 10:48 AM
              • 0 Attachment
                Seems neat, though... do you know if there are any projects out there
                that have given this a whirl?

                Tom

                -----Original Message-----
                From: jeffgrigg63132 [mailto:jgrigg@...]
                Sent: Monday, July 01, 2002 11:52 AM
                To: extremeprogramming@yahoogroups.com
                Subject: [XP] Re: Is it possible to generate code from tests? --
                Evolutionary or Genetic


                --- "Tom Copeland" <tom@i...> wrote:
                > I discussed this with David Dixon-Peugh (http://quilt.sf.net/)
                > and he suggested some sort of genetic algorithm thingy. Since
                > I don't know anything about genetic algorithms, that sounds
                > good to me :-)

                Actually, that's exactly what I was thinking.
                See:
                http://www.c2.com/cgi/wiki?EvolutionaryAlgorithms

                And realize that this approach may be slow, and won't necessarily
                produce the simplest code that can possibly work.


                To Post a message, send it to: extremeprogramming@...

                To Unsubscribe, send a blank message to:
                extremeprogramming-unsubscribe@...

                ad-free courtesy of objectmentor.com

                Your use of Yahoo! Groups is subject to
                http://docs.yahoo.com/info/terms/
              • Kevin Lawrence
                ... From: Tom Copeland To: Sent: Monday, July 01, 2002 10:48 AM Subject: RE: [XP] Re: Is it possible
                Message 7 of 24 , Jul 1 11:30 AM
                • 0 Attachment
                  ----- Original Message -----
                  From: "Tom Copeland" <tom@...>
                  To: <extremeprogramming@yahoogroups.com>
                  Sent: Monday, July 01, 2002 10:48 AM
                  Subject: RE: [XP] Re: Is it possible to generate code from tests? --
                  Evolutionary or Genetic


                  > Seems neat, though... do you know if there are any projects out there
                  > that have given this a whirl?
                  >
                  > Tom

                  I made a start on it about a month ago with the intention of working through
                  the tic-tac-toe exercise that Bill Wake had us playing with as well as the
                  prime factors problem that Uncle Bob posted on the newsgroup. I got as far
                  as building a genetic programming framework and finding solutions to some of
                  the classic GP problems but then I started to run out of Real Life (tm) and
                  put the project on hold for a while.

                  A lot of people raise the issue that the solution would not be TSTTCPW - but
                  I see that as a human concern not a machine concern. If the code can be
                  generated by a computer, what does it matter if a human can read it ? Why
                  would a human ever need to ?

                  Having said that, I also have vague dreams of being able to automatically
                  eliminate duplication from the evolved solution and possibly even attempt
                  more ambitious refactorings automatically.

                  I don't have any illusion that this will replace TDD any time soon, but I am
                  curious whether it is possible - it may even be practical at some point.
                  I'll report back in 15 or 20 years when I have some results.

                  BTW I talked about this on a newsgroup and 3 or 4 other people emailed to
                  say that they are attempting the same thing.

                  Kevin
                • Laurent Bossavit
                  ... And of course if you write the thing test-first it ll eventually bootstrap itself, which is when you ll know the thing works : as long as you have to
                  Message 8 of 24 , Jul 1 11:41 AM
                  • 0 Attachment
                    > I suggested the same thing to Eugene Balyaev, the manager of IntelliJ a
                    > while ago. He said that they'd investigate the idea for v3 of IDEA.

                    And of course if you write the thing test-first it'll eventually
                    bootstrap itself, which is when you'll know the thing works : as long
                    as you have to actually *think*, you're not done yet.

                    Cheers,

                    -[Morendil]-
                    In XP, we don't divide and conquer. We conquer and divide. First we
                    make something that works, then bust it up and solve the little
                    parts.
                    Kent Beck
                  • Dinwiddie, George
                    Tom, If you do this, how will you test the tests? Aren t you just changing the programming language? Is it easier to specify all the behavior in the form of
                    Message 9 of 24 , Jul 1 11:56 AM
                    • 0 Attachment
                      Tom,

                      If you do this, how will you test the tests? Aren't you just changing the
                      programming language? Is it easier to specify all the behavior in the form
                      of tests than in the programming language itself?

                      I like Test Driven Development and find that writing the tests focuses my
                      mind on what I want. I have no illusions that my tests fully specify the
                      behavior, however. There are plenty of small issues that "couldn't possibly
                      break" and have no tests.

                      To me, the verification of testing is the independent evaluation. If the
                      executable code is derived from the test, rather than derived from the
                      understanding of the problem, then it's no longer independent.

                      - George

                      > -----Original Message-----
                      > From: Tom Copeland [mailto:tom@...]
                      > Sent: Monday, July 01, 2002 10:20 AM
                      > To: extremeprogramming@yahoogroups.com
                      > Subject: [XP] Is it possible to generate code from tests?
                      >
                      >
                      > I mean, if you have something like:
                      >
                      > ===================
                      > public void testList() {
                      > List l = new List();
                      > l.add(new Object());
                      > assertTrue(l.size() == 1);
                      > l.removeAll();
                      > assertTrue(l.empty());
                      > }
                      > ===================
                      >
                      > Then could some sort of mutator code generation thingy put together a
                      > List class over a couple of passes like:
                      >
                      > ===================
                      > 1) The test wants a List class. Hm, can't find it in the
                      > classpath, so
                      > I'll generate a List with a no-arg constructor
                      > 2) OK, the test wants a add() that takes an Object, no problem.
                      > 3) OK, the test wants a size() that returns an int, cool.
                      > 4) Hm, so the test failed, so it can't just be any int...
                      > ===================
                      >
                      > etc., etc., I won't continue because I'm not sure what the generator
                      > would do at that point :-)
                      >
                      > If this has already been discussed, sorry about that.
                      > Anybody have any
                      > pointers to any projects like this? Searching Google for "junit test
                      > driven code generation" didn't seem to turn up anything...
                      >
                      > Yours,
                      >
                      > Tom Copeland
                      > InfoEther
                      > 703-486-4543
                      >
                      >
                      > To Post a message, send it to: extremeprogramming@...
                      >
                      > To Unsubscribe, send a blank message to:
                      > extremeprogramming-unsubscribe@...
                      >
                      > ad-free courtesy of objectmentor.com
                      >
                      > Your use of Yahoo! Groups is subject to
                      > http://docs.yahoo.com/info/terms/
                      >
                      >
                    • ejfried@ca.sandia.gov
                      ... I agree. I voiced some enthusiasm for this before, but only as a way to create the minimal stub class after writing a test. Write a test, compile, it
                      Message 10 of 24 , Jul 1 12:17 PM
                      • 0 Attachment
                        I think Dinwiddie, George wrote:

                        > To me, the verification of testing is the independent evaluation. If the
                        > executable code is derived from the test, rather than derived from the
                        > understanding of the problem, then it's no longer independent.
                        >

                        I agree. I voiced some enthusiasm for this before, but only as a way
                        to create the minimal "stub class" after writing a test. Write a test,
                        compile, it won't compile. Generate a stub, compile, now it compiles
                        but fails to run. Now you start editing the generated code. This is
                        just a generalized form of the "Implement Interface" feature many Java
                        IDEs have.


                        -- Ernest Friedman-Hill
                      • jmax017
                        This is an excellent idea. But I think that it would be rather difficult to implement. I have feeling that it would have to work in a similar manner to a
                        Message 11 of 24 , Jul 1 12:32 PM
                        • 0 Attachment
                          This is an excellent idea.
                          But I think that it would be rather difficult to implement.

                          I have feeling that it would have to work in a similar manner to a
                          Functional Programming Languages (SML,Haskell,Scheme etc.) because it
                          is another step away from command oriented lanaguages.

                          I think development of interfaces from tests would be trival (this
                          may exist already) the building of working code would be very
                          difficult.


                          Cheers,
                          Jeff Maxwell
                        • Tom Copeland
                          Hmm.... so if the code could be completely generated from the test, the test would be a formal spec for the code. That s interesting. That s true, when I m
                          Message 12 of 24 , Jul 1 12:33 PM
                          • 0 Attachment
                            Hmm.... so if the code could be completely generated from the test, the
                            test would be a formal spec for the code. That's interesting.

                            That's true, when I'm writing tests, I'm writing them to
                            verify/exercise/use the code. That's why I don't like autogenerated
                            tests, because writing them is a good exercise for figuring out the
                            code.

                            On the other hand, if we could generate a List class from the test, we'd
                            have a pretty sweet deal - a class, a test class, and a bunch of
                            examples on how to use it. Sweet.

                            Yours,

                            Tom

                            -----Original Message-----
                            From: ejfried@... [mailto:ejfried@...]
                            Sent: Monday, July 01, 2002 2:17 PM
                            To: extremeprogramming@yahoogroups.com
                            Subject: Re: [XP] Is it possible to generate code from tests?


                            I think Dinwiddie, George wrote:

                            > To me, the verification of testing is the independent evaluation. If
                            the
                            > executable code is derived from the test, rather than derived from the
                            > understanding of the problem, then it's no longer independent.
                            >

                            I agree. I voiced some enthusiasm for this before, but only as a way
                            to create the minimal "stub class" after writing a test. Write a test,
                            compile, it won't compile. Generate a stub, compile, now it compiles
                            but fails to run. Now you start editing the generated code. This is
                            just a generalized form of the "Implement Interface" feature many Java
                            IDEs have.


                            -- Ernest Friedman-Hill

                            To Post a message, send it to: extremeprogramming@...

                            To Unsubscribe, send a blank message to:
                            extremeprogramming-unsubscribe@...

                            ad-free courtesy of objectmentor.com

                            Your use of Yahoo! Groups is subject to
                            http://docs.yahoo.com/info/terms/
                          • Tom Copeland
                            Right, just like the JavaCC generated parsers. They re gibberish, but who cares? Using something like BCEL (http://jakarta.apache.org/bcel/) would allow the
                            Message 13 of 24 , Jul 1 12:45 PM
                            • 0 Attachment
                              Right, just like the JavaCC generated parsers. They're gibberish, but
                              who cares?

                              Using something like BCEL (http://jakarta.apache.org/bcel/) would allow
                              the "test driven code generator" to generate a test, reload it, and
                              rerun it without having to restart the VM. So it'd be pretty zippy in
                              that regard. Of course, it might still take a million years or so to do
                              :-)

                              Funky stuff!

                              Tom

                              -----Original Message-----
                              From: Kevin Lawrence [mailto:kevin@...]
                              Sent: Monday, July 01, 2002 1:31 PM
                              To: extremeprogramming@yahoogroups.com
                              Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                              Evolutionary or Genetic


                              I made a start on it about a month ago with the intention of working
                              through
                              the tic-tac-toe exercise that Bill Wake had us playing with as well as
                              the
                              prime factors problem that Uncle Bob posted on the newsgroup. I got as
                              far
                              as building a genetic programming framework and finding solutions to
                              some of
                              the classic GP problems but then I started to run out of Real Life (tm)
                              and
                              put the project on hold for a while.

                              > A lot of people raise the issue that the solution would not be TSTTCPW
                              - but
                              > I see that as a human concern not a machine concern. If the code can
                              be
                              > generated by a computer, what does it matter if a human can read it ?
                              Why
                              > would a human ever need to ?

                              Having said that, I also have vague dreams of being able to
                              automatically
                              eliminate duplication from the evolved solution and possibly even
                              attempt
                              more ambitious refactorings automatically.

                              I don't have any illusion that this will replace TDD any time soon, but
                              I am
                              curious whether it is possible - it may even be practical at some point.
                              I'll report back in 15 or 20 years when I have some results.

                              BTW I talked about this on a newsgroup and 3 or 4 other people emailed
                              to
                              say that they are attempting the same thing.

                              Kevin




                              To Post a message, send it to: extremeprogramming@...

                              To Unsubscribe, send a blank message to:
                              extremeprogramming-unsubscribe@...

                              ad-free courtesy of objectmentor.com

                              Your use of Yahoo! Groups is subject to
                              http://docs.yahoo.com/info/terms/
                            • Richard Faust
                              Constraint Programming, maybe? Constraint programming represents one of the closest approaches computer science has yet made to the Holy Grail of programming:
                              Message 14 of 24 , Jul 1 1:14 PM
                              • 0 Attachment
                                Constraint Programming, maybe?

                                "Constraint programming represents one of the closest approaches computer
                                science has yet made to the Holy Grail of programming: the user states the
                                problem, the computer solves it."
                                Eugene C. Freuder, April 1997
                                http://www.kluweronline.com/issn/1383-7133

                                At 19:32 1/7/2002 +0000, you wrote:
                                >This is an excellent idea.
                                >But I think that it would be rather difficult to implement.
                                >
                                >I have feeling that it would have to work in a similar manner to a
                                >Functional Programming Languages (SML,Haskell,Scheme etc.) because it
                                >is another step away from command oriented lanaguages.
                                >
                                >I think development of interfaces from tests would be trival (this
                                >may exist already) the building of working code would be very
                                >difficult.
                                >
                                >
                                >Cheers,
                                >Jeff Maxwell
                                >
                                >
                                >
                                >
                                >
                                >
                                >
                                >
                                >To Post a message, send it to: extremeprogramming@...
                                >
                                >To Unsubscribe, send a blank message to:
                                >extremeprogramming-unsubscribe@...
                                >
                                >ad-free courtesy of objectmentor.com
                                >
                                >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                              • jmax017
                                Yes I imagine that would be similar. Test are just a method of stating the problem. Cheers, Jeff Maxwell ... computer ... states the ... it ...
                                Message 15 of 24 , Jul 1 2:13 PM
                                • 0 Attachment
                                  Yes I imagine that would be similar.
                                  Test are just a method of stating the problem.

                                  Cheers,
                                  Jeff Maxwell

                                  --- In extremeprogramming@y..., Richard Faust <richard.faust@i...>
                                  wrote:
                                  > Constraint Programming, maybe?
                                  >
                                  > "Constraint programming represents one of the closest approaches
                                  computer
                                  > science has yet made to the Holy Grail of programming: the user
                                  states the
                                  > problem, the computer solves it."
                                  > Eugene C. Freuder, April 1997
                                  > http://www.kluweronline.com/issn/1383-7133
                                  >
                                  > At 19:32 1/7/2002 +0000, you wrote:
                                  > >This is an excellent idea.
                                  > >But I think that it would be rather difficult to implement.
                                  > >
                                  > >I have feeling that it would have to work in a similar manner to a
                                  > >Functional Programming Languages (SML,Haskell,Scheme etc.) because
                                  it
                                  > >is another step away from command oriented lanaguages.
                                  > >
                                  > >I think development of interfaces from tests would be trival (this
                                  > >may exist already) the building of working code would be very
                                  > >difficult.
                                  > >
                                  > >
                                  > >Cheers,
                                  > >Jeff Maxwell
                                  > >
                                  > >
                                  > >
                                  > >
                                  > >
                                  > >
                                  > >
                                  > >
                                  > >To Post a message, send it to: extremeprogramming@e...
                                  > >
                                  > >To Unsubscribe, send a blank message to:
                                  > >extremeprogramming-unsubscribe@e...
                                  > >
                                  > >ad-free courtesy of objectmentor.com
                                  > >
                                  > >Your use of Yahoo! Groups is subject to
                                  http://docs.yahoo.com/info/terms/
                                • jmax017
                                  Correct something Like javaCC. JavaCC has a very nice advantage of using grammers to build its parsers and grammers are nice and mathametical. Can your test be
                                  Message 16 of 24 , Jul 1 2:29 PM
                                  • 0 Attachment
                                    Correct something Like javaCC.

                                    JavaCC has a very nice advantage of using grammers to build its
                                    parsers and grammers are nice and mathametical.

                                    Can your test be as mathamatical?

                                    Cheers,
                                    Jeff Maxwell



                                    --- In extremeprogramming@y..., "Tom Copeland" <tom@i...> wrote:
                                    > Right, just like the JavaCC generated parsers. They're gibberish,
                                    but
                                    > who cares?
                                    >
                                    > Using something like BCEL (http://jakarta.apache.org/bcel/) would
                                    allow
                                    > the "test driven code generator" to generate a test, reload it, and
                                    > rerun it without having to restart the VM. So it'd be pretty zippy
                                    in
                                    > that regard. Of course, it might still take a million years or so
                                    to do
                                    > :-)
                                    >
                                    > Funky stuff!
                                    >
                                    > Tom
                                    >
                                    > -----Original Message-----
                                    > From: Kevin Lawrence [mailto:kevin@d...]
                                    > Sent: Monday, July 01, 2002 1:31 PM
                                    > To: extremeprogramming@y...
                                    > Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                                    > Evolutionary or Genetic
                                    >
                                    >
                                    > I made a start on it about a month ago with the intention of working
                                    > through
                                    > the tic-tac-toe exercise that Bill Wake had us playing with as well
                                    as
                                    > the
                                    > prime factors problem that Uncle Bob posted on the newsgroup. I got
                                    as
                                    > far
                                    > as building a genetic programming framework and finding solutions to
                                    > some of
                                    > the classic GP problems but then I started to run out of Real Life
                                    (tm)
                                    > and
                                    > put the project on hold for a while.
                                    >
                                    > > A lot of people raise the issue that the solution would not be
                                    TSTTCPW
                                    > - but
                                    > > I see that as a human concern not a machine concern. If the code
                                    can
                                    > be
                                    > > generated by a computer, what does it matter if a human can read
                                    it ?
                                    > Why
                                    > > would a human ever need to ?
                                    >
                                    > Having said that, I also have vague dreams of being able to
                                    > automatically
                                    > eliminate duplication from the evolved solution and possibly even
                                    > attempt
                                    > more ambitious refactorings automatically.
                                    >
                                    > I don't have any illusion that this will replace TDD any time soon,
                                    but
                                    > I am
                                    > curious whether it is possible - it may even be practical at some
                                    point.
                                    > I'll report back in 15 or 20 years when I have some results.
                                    >
                                    > BTW I talked about this on a newsgroup and 3 or 4 other people
                                    emailed
                                    > to
                                    > say that they are attempting the same thing.
                                    >
                                    > Kevin
                                    >
                                    >
                                    >
                                    >
                                    > To Post a message, send it to: extremeprogramming@e...
                                    >
                                    > To Unsubscribe, send a blank message to:
                                    > extremeprogramming-unsubscribe@e...
                                    >
                                    > ad-free courtesy of objectmentor.com
                                    >
                                    > Your use of Yahoo! Groups is subject to
                                    > http://docs.yahoo.com/info/terms/
                                  • Tom Copeland
                                    Yup, true, my unit tests usually aren t nearly as precise as an EBNF grammer. Still though, in some cases I ve ended up with a suite of tests that specified a
                                    Message 17 of 24 , Jul 1 2:43 PM
                                    • 0 Attachment
                                      Yup, true, my unit tests usually aren't nearly as precise as an EBNF
                                      grammer.

                                      Still though, in some cases I've ended up with a suite of tests that
                                      specified a class' behavior with pretty good preciseness - at least, I
                                      think it did. I dunno.... I kind of feel like tests would overlap in
                                      ways that would cover a lot of ground... and an implementation that
                                      passed all the tests would be pretty good.

                                      Of course, that gets us back to: even if the tests are quite
                                      comprehensive, how does the code generator put them together to make a
                                      real Java class? And thence into the depths of all sorts of genetic
                                      programming niftiness. Cool stuff to think about though....

                                      Yours,

                                      Tom

                                      -----Original Message-----
                                      From: jmax017 [mailto:jmax01@...]
                                      Sent: Monday, July 01, 2002 4:30 PM
                                      To: extremeprogramming@yahoogroups.com
                                      Subject: [XP] Re: Is it possible to generate code from tests? --
                                      Evolutionary or Genetic


                                      Correct something Like javaCC.

                                      JavaCC has a very nice advantage of using grammers to build its
                                      parsers and grammers are nice and mathametical.

                                      Can your test be as mathamatical?

                                      Cheers,
                                      Jeff Maxwell

                                      <snip>
                                    • hpforever
                                      IDEA can generate method stub from its usage. E.g. from the Tom test case below following class can be generated automagically: public class List { public
                                      Message 18 of 24 , Jul 2 1:05 AM
                                      • 0 Attachment
                                        IDEA can generate method stub from its usage.
                                        E.g. from the Tom' test case below following class can be generated
                                        automagically:
                                        public class List {
                                        public void add(Object o) {}
                                        public void removeAll() {}
                                        public int size() { return 0; }
                                        public boolean empty() { return false; }
                                        }
                                        As for realizing add() and size() relation and figuring out
                                        corresponding algorithms, I afraid, it is Universal Problem Solver
                                        job.
                                        http://www.personal.leeds.ac.uk/~ecl6nb/OnTarget/1989/war/war92cov.ht
                                        m

                                        --
                                        regards,
                                        Alexey Kudravtsev
                                        cdrATintellij.com

                                        --- In extremeprogramming@y..., "Bryan Dollery" <Bryan.Dollery@C...>
                                        wrote:
                                        > Hi Tom,
                                        >
                                        > I suggested the same thing to Eugene Balyaev, the manager of
                                        IntelliJ a
                                        > while ago. He said that they'd investigate the idea for v3 of IDEA.
                                        >
                                        > Cheers,
                                        >
                                        > Bryan
                                        >
                                        >
                                        > > -----Original Message-----
                                        > > From: Tom Copeland [mailto:tom@i...]
                                        > > Sent: Tuesday, July 02, 2002 2:20 AM
                                        > > To: extremeprogramming@y...
                                        > > Subject: [XP] Is it possible to generate code from tests?
                                        > >
                                        > >
                                        > > I mean, if you have something like:
                                        > >
                                        > > ===================
                                        > > public void testList() {
                                        > > List l = new List();
                                        > > l.add(new Object());
                                        > > assertTrue(l.size() == 1);
                                        > > l.removeAll();
                                        > > assertTrue(l.empty());
                                        > > }
                                        > > ===================
                                        > >
                                        > > Then could some sort of mutator code generation thingy put
                                        together a
                                        > > List class over a couple of passes like:
                                        > >
                                        > > ===================
                                        > > 1) The test wants a List class. Hm, can't find it in the
                                        classpath, so
                                        > > I'll generate a List with a no-arg constructor
                                        > > 2) OK, the test wants a add() that takes an Object, no problem.
                                        > > 3) OK, the test wants a size() that returns an int, cool.
                                        > > 4) Hm, so the test failed, so it can't just be any int...
                                        > > ===================
                                        > >
                                        > > etc., etc., I won't continue because I'm not sure what the
                                        generator
                                        > > would do at that point :-)
                                        > >
                                        > > If this has already been discussed, sorry about that. Anybody
                                        have any
                                        > > pointers to any projects like this? Searching Google for "junit
                                        test
                                        > > driven code generation" didn't seem to turn up anything...
                                        > >
                                      • Robert Blum
                                        Hi Tom, ... Would you still think that if you renamed all the variables and methods to foo(), bar, and similar names? Serious question. At least for me, a good
                                        Message 19 of 24 , Jul 2 7:18 AM
                                        • 0 Attachment
                                          Hi Tom,

                                          >Still though, in some cases I've ended up with a suite of tests that
                                          >specified a class' behavior with pretty good preciseness - at least, I
                                          >think it did.
                                          >
                                          Would you still think that if you renamed all the variables and methods
                                          to foo(), bar, and similar
                                          names? Serious question. At least for me, a good part of the value of a
                                          test is in its readability.
                                          I /know/ what add() and size() (should) do - the code generator might
                                          have a problem.

                                          >I dunno.... I kind of feel like tests would overlap in
                                          >ways that would cover a lot of ground... and an implementation that
                                          >passed all the tests would be pretty good.
                                          >
                                          One problem would be to determine if pretty good is sufficiently good.
                                          As we've seen in the
                                          'binary search' thread, it's quite easy to create an algorithm that
                                          works most of the time. :)

                                          We can then use tests to explore the areas where we are not sure if the
                                          code is behaving
                                          properly: boundary conditions, for example. With automatically generated
                                          code, that might
                                          be more difficult to do, simply because we might not be able to quickly
                                          understand what
                                          the code does.

                                          Nevertheless, it sounds like a very interesting thing to have. I wonder
                                          if there's any research
                                          on the topic....


                                          - Robert
                                        • Tom Copeland
                                          Yeah, the variable name thing is something. Here s an excerpt fm JavaParser (generated by JavaCC): =========== if (jj_2_30(2147483647)) {
                                          Message 20 of 24 , Jul 2 7:33 AM
                                          • 0 Attachment
                                            Yeah, the variable name thing is something. Here's an excerpt fm
                                            JavaParser (generated by JavaCC):

                                            ===========
                                            if (jj_2_30(2147483647)) {
                                            LocalVariableDeclaration();
                                            } else {
                                            switch ((jj_ntk==-1)?jj_ntk():jj_ntk) {
                                            ===========

                                            Yikes. On the other hand, it's not meant to be read, and the parts that
                                            are meant to be read - i.e., the grammer - are much clearer.

                                            If the generated code passed the tests, though, all the externally
                                            visible symbols - function names and whatnot - would all have reasonable
                                            names, because they'd be specified by the test. Internally it might be
                                            gibberish... but since that would be private and local variables, maybe
                                            that's not a big deal?

                                            My wife is a mathematician and, when I talked to her about this,
                                            immediately suggested recurrence relations. So the computer is given
                                            "1,3,5,7" and has to come up with "2n+1" as the solution. I'm not sure
                                            how that would translate to code, though... there are a lot more
                                            operators and operands to work with.

                                            Maybe it's some sort of neural net thingy, where you seed the net with a
                                            bunch of sample patterns - i.e., a put() followed by a get() implies an
                                            instance variable to maintain state. Maybe you could feed the JDK
                                            source into the neural net and that would be the learning mechanism.
                                            Maybe that sounds reasonable to me just because I know nothing
                                            whatsoever about neural nets :-)

                                            Yours,

                                            Tom

                                            -----Original Message-----
                                            From: Robert Blum [mailto:r.blum@...]
                                            Sent: Tuesday, July 02, 2002 9:18 AM
                                            To: extremeprogramming@yahoogroups.com
                                            Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                                            Evolutionary or Genetic


                                            Hi Tom,

                                            >Still though, in some cases I've ended up with a suite of tests that
                                            >specified a class' behavior with pretty good preciseness - at least, I
                                            >think it did.
                                            >
                                            Would you still think that if you renamed all the variables and methods
                                            to foo(), bar, and similar
                                            names? Serious question. At least for me, a good part of the value of a
                                            test is in its readability.
                                            I /know/ what add() and size() (should) do - the code generator might
                                            have a problem.

                                            >I dunno.... I kind of feel like tests would overlap in
                                            >ways that would cover a lot of ground... and an implementation that
                                            >passed all the tests would be pretty good.
                                            >
                                            One problem would be to determine if pretty good is sufficiently good.
                                            As we've seen in the
                                            'binary search' thread, it's quite easy to create an algorithm that
                                            works most of the time. :)

                                            We can then use tests to explore the areas where we are not sure if the
                                            code is behaving
                                            properly: boundary conditions, for example. With automatically generated

                                            code, that might
                                            be more difficult to do, simply because we might not be able to quickly
                                            understand what
                                            the code does.

                                            Nevertheless, it sounds like a very interesting thing to have. I wonder
                                            if there's any research
                                            on the topic....


                                            - Robert


                                            To Post a message, send it to: extremeprogramming@...

                                            To Unsubscribe, send a blank message to:
                                            extremeprogramming-unsubscribe@...

                                            ad-free courtesy of objectmentor.com

                                            Your use of Yahoo! Groups is subject to
                                            http://docs.yahoo.com/info/terms/
                                          • Griffin Caprio
                                            I think it would be possible to at least generate the signatures. After all, many UML tools can reverse engineer/generate code. Griffin ... Griffin Caprio
                                            Message 21 of 24 , Jul 2 2:54 PM
                                            • 0 Attachment
                                              I think it would be possible to at least generate the signatures. After
                                              all, many UML tools can reverse engineer/generate code.

                                              Griffin


                                              >From: ejfried@...
                                              >Reply-To: extremeprogramming@yahoogroups.com
                                              >To: extremeprogramming@yahoogroups.com
                                              >Subject: Re: [XP] Is it possible to generate code from tests?
                                              >Date: Mon, 1 Jul 2002 08:15:47 -0700 (PDT)
                                              >
                                              >I think Tom Copeland wrote:
                                              > >
                                              > > ===================
                                              > > 1) The test wants a List class. Hm, can't find it in the classpath, so
                                              > > I'll generate a List with a no-arg constructor
                                              > > 2) OK, the test wants a add() that takes an Object, no problem.
                                              > > 3) OK, the test wants a size() that returns an int, cool.
                                              > > 4) Hm, so the test failed, so it can't just be any int...
                                              > > ===================
                                              > >
                                              > > etc., etc., I won't continue because I'm not sure what the generator
                                              > > would do at that point :-)
                                              > >
                                              >
                                              >This sounds useful, although I'm pretty sure at the point you stopped
                                              >is where it becomes an impossible mind-reading problem, at least in
                                              >the general case. It'd still be a nice accelerator for TDD. So,
                                              >anybody ever tried this?
                                              >
                                              >
                                              >---------------------------------------------------------
                                              >Ernest Friedman-Hill
                                              >Distributed Systems Research Phone: (925) 294-2154
                                              >Sandia National Labs FAX: (925) 294-2234
                                              >Org. 8920, MS 9012 ejfried@...
                                              >PO Box 969 http://herzberg.ca.sandia.gov
                                              >Livermore, CA 94550
                                              >
                                              >To Post a message, send it to: extremeprogramming@...
                                              >
                                              >To Unsubscribe, send a blank message to:
                                              >extremeprogramming-unsubscribe@...
                                              >
                                              >ad-free courtesy of objectmentor.com
                                              >
                                              >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/




                                              Griffin Caprio
                                              cell: (773) 230-0936


                                              _________________________________________________________________
                                              Send and receive Hotmail on your mobile device: http://mobile.msn.com
                                            • Robert Blum
                                              Hi Tom, ... [snip] ... Ah, I was not talking about the generated code. Most people tend to just live with (compiler-) generated code and never look at them.
                                              Message 22 of 24 , Jul 6 11:44 AM
                                              • 0 Attachment
                                                Hi Tom,

                                                >Yeah, the variable name thing is something. Here's an excerpt fm
                                                >JavaParser (generated by JavaCC):
                                                >
                                                [snip]

                                                >Yikes. On the other hand, it's not meant to be read, and the parts that
                                                >are meant to be read - i.e., the grammer - are much clearer.

                                                Ah, I was not talking about the generated code. Most people tend to just
                                                live with
                                                (compiler-) generated code and never look at them. But the test cases
                                                carry extra
                                                information in the names of the functions and variables they use that is
                                                inaccessible
                                                to the automated code generator.

                                                Unless the code generator is also able to understand human language.
                                                E.g. if I write
                                                a test for a function called EstimateDmaTransferSize, I know just by
                                                looking at the
                                                name what the function is supposed to do.

                                                But this requires knowledge about what a DMA transfer is, and how you
                                                estimate
                                                its size for the platform in question. Nothing an automated code
                                                generator in the
                                                near future will have.

                                                >
                                                >gibberish... but since that would be private and local variables, maybe
                                                >that's not a big deal?
                                                >
                                                It is not a big deal. The big deal is in the information contained in
                                                the public name, yet
                                                not accessible to executable code.

                                                >My wife is a mathematician and, when I talked to her about this,
                                                >immediately suggested recurrence relations. So the computer is given
                                                >"1,3,5,7" and has to come up with "2n+1" as the solution.
                                                >
                                                Yes, but what if my function is named GenerateAllPrimesIncludingOne? Of
                                                course,
                                                you can argue that in this case my test case is not complete. But as
                                                long as you do
                                                not cover a functions output exhaustively, there will always be a
                                                function that does
                                                /almost/ the same thing as what you intended.

                                                Unit Tests are never complete. They're a safety net for our
                                                refactorings, a design
                                                tool, and a cure for our fear of breaking things. But they are not a
                                                complete
                                                specification of the system. Not even if you include ATs.

                                                >Maybe it's some sort of neural net thingy, where you seed the net with a
                                                >bunch of sample patterns - i.e., a put() followed by a get() implies an
                                                >instance variable to maintain state. Maybe you could feed the JDK
                                                >source into the neural net and that would be the learning mechanism.
                                                >Maybe that sounds reasonable to me just because I know nothing
                                                >whatsoever about neural nets :-)
                                                >
                                                Since I have only limited knowledge of NNs myself, take the following
                                                with a
                                                couple of salt shakers: (And I'm sure there's somebody here on the list
                                                who can
                                                provide enlightnment for us)

                                                AFAIK, a NN is good at pattern recognition. That means that a neural
                                                net could
                                                help you recognize that you're trying to build something that has been
                                                built before.

                                                So it might ( or might not) provide some guidance - but unless all
                                                programs follow
                                                the same set of patterns (some of them hidden from view), there's not
                                                much to be
                                                gained. NNs seem to be rather bad at coming up with new, inventive
                                                solutions.

                                                And their main drawback is that they don't always do what you expect
                                                them to do.
                                                Bad training data has bad results. Your above NN might have a tendency
                                                to re-
                                                write the JDK time and again. (Hey, is maybe SUN using a NN? :)

                                                - Robert
                                              • Tom Copeland
                                                Yup, after scanning a bunch of genetic programming stuff at the local MicroCenter (Lee Highway, northern VA, has an excellent technical library), I have to
                                                Message 23 of 24 , Jul 8 6:33 AM
                                                • 0 Attachment
                                                  Yup, after scanning a bunch of genetic programming stuff at the local
                                                  MicroCenter (Lee Highway, northern VA, has an excellent technical
                                                  library), I have to agree. Generating an implementation from unit tests
                                                  seems to 1) be impossible and 2) be worthless even if it was possible.
                                                  Ah well.... fun to think about anyhow...

                                                  Yours,

                                                  Tom

                                                  -----Original Message-----
                                                  From: Robert Blum [mailto:r.blum@...]
                                                  Sent: Saturday, July 06, 2002 1:44 PM
                                                  To: extremeprogramming@yahoogroups.com
                                                  Subject: Re: [XP] Re: Is it possible to generate code from tests? --
                                                  Evolutionary or Genetic


                                                  Hi Tom,

                                                  >Yeah, the variable name thing is something. Here's an excerpt fm
                                                  >JavaParser (generated by JavaCC):
                                                  >
                                                  [snip]

                                                  >Yikes. On the other hand, it's not meant to be read, and the parts
                                                  that
                                                  >are meant to be read - i.e., the grammer - are much clearer.

                                                  Ah, I was not talking about the generated code. Most people tend to just

                                                  live with
                                                  (compiler-) generated code and never look at them. But the test cases
                                                  carry extra
                                                  information in the names of the functions and variables they use that is

                                                  inaccessible
                                                  to the automated code generator.

                                                  Unless the code generator is also able to understand human language.
                                                  E.g. if I write
                                                  a test for a function called EstimateDmaTransferSize, I know just by
                                                  looking at the
                                                  name what the function is supposed to do.

                                                  But this requires knowledge about what a DMA transfer is, and how you
                                                  estimate
                                                  its size for the platform in question. Nothing an automated code
                                                  generator in the
                                                  near future will have.

                                                  >
                                                  >gibberish... but since that would be private and local variables, maybe
                                                  >that's not a big deal?
                                                  >
                                                  It is not a big deal. The big deal is in the information contained in
                                                  the public name, yet
                                                  not accessible to executable code.

                                                  >My wife is a mathematician and, when I talked to her about this,
                                                  >immediately suggested recurrence relations. So the computer is given
                                                  >"1,3,5,7" and has to come up with "2n+1" as the solution.
                                                  >
                                                  Yes, but what if my function is named GenerateAllPrimesIncludingOne? Of

                                                  course,
                                                  you can argue that in this case my test case is not complete. But as
                                                  long as you do
                                                  not cover a functions output exhaustively, there will always be a
                                                  function that does
                                                  /almost/ the same thing as what you intended.

                                                  Unit Tests are never complete. They're a safety net for our
                                                  refactorings, a design
                                                  tool, and a cure for our fear of breaking things. But they are not a
                                                  complete
                                                  specification of the system. Not even if you include ATs.

                                                  >Maybe it's some sort of neural net thingy, where you seed the net with
                                                  a
                                                  >bunch of sample patterns - i.e., a put() followed by a get() implies an
                                                  >instance variable to maintain state. Maybe you could feed the JDK
                                                  >source into the neural net and that would be the learning mechanism.
                                                  >Maybe that sounds reasonable to me just because I know nothing
                                                  >whatsoever about neural nets :-)
                                                  >
                                                  Since I have only limited knowledge of NNs myself, take the following
                                                  with a
                                                  couple of salt shakers: (And I'm sure there's somebody here on the list
                                                  who can
                                                  provide enlightnment for us)

                                                  AFAIK, a NN is good at pattern recognition. That means that a neural
                                                  net could
                                                  help you recognize that you're trying to build something that has been
                                                  built before.

                                                  So it might ( or might not) provide some guidance - but unless all
                                                  programs follow
                                                  the same set of patterns (some of them hidden from view), there's not
                                                  much to be
                                                  gained. NNs seem to be rather bad at coming up with new, inventive
                                                  solutions.

                                                  And their main drawback is that they don't always do what you expect
                                                  them to do.
                                                  Bad training data has bad results. Your above NN might have a tendency
                                                  to re-
                                                  write the JDK time and again. (Hey, is maybe SUN using a NN? :)

                                                  - Robert


                                                  To Post a message, send it to: extremeprogramming@...

                                                  To Unsubscribe, send a blank message to:
                                                  extremeprogramming-unsubscribe@...

                                                  ad-free courtesy of objectmentor.com

                                                  Your use of Yahoo! Groups is subject to
                                                  http://docs.yahoo.com/info/terms/
                                                Your message has been successfully submitted and would be delivered to recipients shortly.