Loading ...
Sorry, an error occurred while loading the content.

ANTLR translation - e.g. Java->tree->Java

Expand Messages
  • Tim Tyler
    I m contemplating an ANTLR translation project. To do this I need to reconstruct the Java file from its tree. There seem to be several problems. One is that
    Message 1 of 9 , Jun 22, 2003
    • 0 Attachment
      I'm contemplating an ANTLR translation project.

      To do this I need to reconstruct the Java file from its tree.

      There seem to be several problems.

      One is that unicode information seems to get blatted.

      Another is that most of the Java parsers I can see
      strip out comments and white space. I'm likely to
      be wanting to preserve these as much as possible -
      and maybe have them in the parse tree.

      About the only existing Java ANTLR translation project
      I'm aware of is Jalopy: http://jalopy.sourceforge.net/

      Jalopy suggests that something is at least possible - but
      it is big and complex - and I'd like a bit of a less hairy
      example.

      Are there any other ANTLR translation projects out there?

      I'm a bit suprised ANTLR isn't already a little more translation-
      oriented.

      After all, being able to reconstruct the original file is a good
      test for having parsed it correctly ;-)
    • paul taney
      ... Java ...parsers I can see ... I was attempting something similar and adapted code from antlr-2.7.2/examples/java/preserveWhiteSpace in the following way
      Message 2 of 9 , Jun 22, 2003
      • 0 Attachment
        --- Tim Tyler <tt2333@...> wrote:most of the
        Java > ...parsers I can see
        > strip out comments and white space. I'm likely to
        > be wanting to preserve these as much as possible -
        > and maybe have them in the parse tree.
        >

        I was attempting something similar and adapted
        code from

        antlr-2.7.2/examples/java/preserveWhiteSpace

        in the following way

        public static void main(String[] args) {
        JavaLexer lexer = new JavaLexer(new
        DataInputStream(System.in));

        lexer.setTokenObjectClass("antlr.CommonHiddenStreamToken");
        filter = new TokenStreamHiddenTokenFilter(lexer);
        filter.hide(JavaRecognizer.WS);
        filter.hide(JavaRecognizer.SL_COMMENT);
        JavaRecognizer parser = new JavaRecognizer(filter);

        parser.setASTNodeClass("antlr.CommonASTWithHiddenTokens");
        try {
        //rewire the next line for java's root node
        parser.compilationUnit();
        }

        I'm still working it out, as this is my first antlr
        project. But I think this is on the right track...

        Feedback would be appreciated.

        paul taney


        __________________________________
        Do you Yahoo!?
        SBC Yahoo! DSL - Now only $29.95 per month!
        http://sbc.yahoo.com
      • Matt Benson
        I once saw an educational project that was at least influenced by Ter and some other folks, that built symbol tables and all. Also, the Transmogrify project
        Message 3 of 9 , Jun 23, 2003
        • 0 Attachment
          I once saw an educational project that was at least
          influenced by Ter and some other folks, that built
          symbol tables and all. Also, the Transmogrify project
          on sourceforge used ANTLR for refactoring. I spent
          several months bastardizing its code to my liking for
          a code-completion tool but have never finished the
          project. I took a huge break from what I was working
          on because I just didn't understand what was going on
          with ANTLR. So I followed Ter's USF course as much as
          possible, and I've just finished a 2000+ line parser
          grammar file that actually works so I think I must
          have gained some useful knowledge. Next I have to
          master lexer and treeparser grammars.

          If all you want to do is reconstitute a Java file from
          the AST it sounds like Jalopy would be one of the
          simplest options--projects that build symbol tables
          are probably more complex than you would want to get
          tangled up with.

          -Matt

          --- Tim Tyler <tt2333@...> wrote:
          > I'm contemplating an ANTLR translation project.
          >
          > To do this I need to reconstruct the Java file from
          > its tree.
          >
          > There seem to be several problems.
          >
          > One is that unicode information seems to get
          > blatted.
          >
          > Another is that most of the Java parsers I can see
          > strip out comments and white space. I'm likely to
          > be wanting to preserve these as much as possible -
          > and maybe have them in the parse tree.
          >
          > About the only existing Java ANTLR translation
          > project
          > I'm aware of is Jalopy:
          > http://jalopy.sourceforge.net/
          >
          > Jalopy suggests that something is at least possible
          > - but
          > it is big and complex - and I'd like a bit of a less
          > hairy
          > example.
          >
          > Are there any other ANTLR translation projects out
          > there?
          >
          > I'm a bit suprised ANTLR isn't already a little more
          > translation-
          > oriented.
          >
          > After all, being able to reconstruct the original
          > file is a good
          > test for having parsed it correctly ;-)
          >
          >
          >
          >
          >
          > Your use of Yahoo! Groups is subject to
          > http://docs.yahoo.com/info/terms/
          >
          >


          __________________________________
          Do you Yahoo!?
          SBC Yahoo! DSL - Now only $29.95 per month!
          http://sbc.yahoo.com
        • mzukowski@yci.com
          There are many different reasons to go from source to tree to source. How extensively you want to manipulate the tree will dictate how well you can preserve
          Message 4 of 9 , Jun 24, 2003
          • 0 Attachment
            There are many different reasons to go from source to tree to source. How
            extensively you want to manipulate the tree will dictate how well you can
            preserve whitespace and comments.

            Sometimes comments are attached to the preceeding construct, sometimes the
            following. Sometimes it depends on if it's on the same line or not.
            Sometimes expressions get special formatting and sometimes they don't.
            Sometimes you delete, sometimes you rearrange, sometimes you just replace
            the text in the node.

            There is no easy general approach to preserving whitespace and comments. If
            you're not rearranging the tree then it's trivial. Otherwise there are lots
            of semantics to work out.

            Monty

            -----Original Message-----
            From: Tim Tyler [mailto:tt2333@...]
            Sent: Sunday, June 22, 2003 5:44 AM
            To: antlr-interest@yahoogroups.com
            Subject: [antlr-interest] ANTLR translation - e.g. Java->tree->Java


            I'm contemplating an ANTLR translation project.

            To do this I need to reconstruct the Java file from its tree.

            There seem to be several problems.

            One is that unicode information seems to get blatted.

            Another is that most of the Java parsers I can see
            strip out comments and white space. I'm likely to
            be wanting to preserve these as much as possible -
            and maybe have them in the parse tree.

            About the only existing Java ANTLR translation project
            I'm aware of is Jalopy: http://jalopy.sourceforge.net/

            Jalopy suggests that something is at least possible - but
            it is big and complex - and I'd like a bit of a less hairy example.

            Are there any other ANTLR translation projects out there?

            I'm a bit suprised ANTLR isn't already a little more translation- oriented.

            After all, being able to reconstruct the original file is a good
            test for having parsed it correctly ;-)





            Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
          • atripp54321
            If you want more details on the issues of preserving whitespace and comments, here s a good article:
            Message 5 of 9 , Jun 24, 2003
            • 0 Attachment
              If you want more details on the issues of preserving whitespace
              and comments, here's a good article:
              http://research.sun.com/projects/jackpot/COM.sun.mlvdv.doc.scam_nov01.paper_pdf.pdf



              --- In antlr-interest@yahoogroups.com, mzukowski@y... wrote:
              > There are many different reasons to go from source to tree to
              source. How
              > extensively you want to manipulate the tree will dictate how well
              you can
              > preserve whitespace and comments.
              >
              > Sometimes comments are attached to the preceeding construct,
              sometimes the
              > following. Sometimes it depends on if it's on the same line or not.
              > Sometimes expressions get special formatting and sometimes they don't.
              > Sometimes you delete, sometimes you rearrange, sometimes you just
              replace
              > the text in the node.
              >
              > There is no easy general approach to preserving whitespace and
              comments. If
              > you're not rearranging the tree then it's trivial. Otherwise there
              are lots
              > of semantics to work out.
              >
              > Monty
              >
              > -----Original Message-----
              > From: Tim Tyler [mailto:tt2333@y...]
              > Sent: Sunday, June 22, 2003 5:44 AM
              > To: antlr-interest@yahoogroups.com
              > Subject: [antlr-interest] ANTLR translation - e.g. Java->tree->Java
              >
              >
              > I'm contemplating an ANTLR translation project.
              >
              > To do this I need to reconstruct the Java file from its tree.
              >
              > There seem to be several problems.
              >
              > One is that unicode information seems to get blatted.
              >
              > Another is that most of the Java parsers I can see
              > strip out comments and white space. I'm likely to
              > be wanting to preserve these as much as possible -
              > and maybe have them in the parse tree.
              >
              > About the only existing Java ANTLR translation project
              > I'm aware of is Jalopy: http://jalopy.sourceforge.net/
              >
              > Jalopy suggests that something is at least possible - but
              > it is big and complex - and I'd like a bit of a less hairy example.
              >
              > Are there any other ANTLR translation projects out there?
              >
              > I'm a bit suprised ANTLR isn't already a little more translation-
              oriented.
              >
              > After all, being able to reconstruct the original file is a good
              > test for having parsed it correctly ;-)
              >
              >
              >
              >
              >
              > Your use of Yahoo! Groups is subject to
              http://docs.yahoo.com/info/terms/
            • atripp54321
              I have a straightforward JavaEmitter class that produces decent-looking java code from an ANTLR AST. I ll mail it to you (I assume it s poor edicate to post
              Message 6 of 9 , Jun 24, 2003
              • 0 Attachment
                I have a straightforward JavaEmitter class that produces
                decent-looking java code from an ANTLR AST.
                I'll mail it to you (I assume it's poor edicate to post
                900 lines of code).

                Andy

                --- In antlr-interest@yahoogroups.com, "Tim Tyler" <tt2333@y...>
                wrote:
                > I'm contemplating an ANTLR translation project.
                >
                > To do this I need to reconstruct the Java file from its tree.
                >
                > There seem to be several problems.
                >
                > One is that unicode information seems to get blatted.
                >
                > Another is that most of the Java parsers I can see
                > strip out comments and white space. I'm likely to
                > be wanting to preserve these as much as possible -
                > and maybe have them in the parse tree.
                >
                > About the only existing Java ANTLR translation project
                > I'm aware of is Jalopy: http://jalopy.sourceforge.net/
                >
                > Jalopy suggests that something is at least possible - but
                > it is big and complex - and I'd like a bit of a less hairy
                > example.
                >
                > Are there any other ANTLR translation projects out there?
                >
                > I'm a bit suprised ANTLR isn't already a little more translation-
                > oriented.
                >
                > After all, being able to reconstruct the original file is a good
                > test for having parsed it correctly ;-)
              • Anthony Brenelière
                The recognition exception is in a private member named recog, I need to get the ligne number for that exception and I don t know how to do. Is there a way to
                Message 7 of 9 , Jun 25, 2003
                • 0 Attachment
                  The recognition exception is in a private member named recog,

                  I need to get the ligne number for that exception and I don't know how
                  to do.

                  Is there a way to solve that problem ?

                  Cordially,
                  Anthony Breneliere
                • Tim Tyler
                  ... Interesting - thanks. I wouldn t describe it as straight-forward ;-)
                  Message 8 of 9 , Jun 25, 2003
                  • 0 Attachment
                    --- "atripp54321" <atripp@c...> wrote:

                    > I have a straightforward JavaEmitter class that produces
                    > decent-looking java code from an ANTLR AST.
                    > I'll mail it to you (I assume it's poor edicate to post
                    > 900 lines of code).

                    Interesting - thanks.

                    I wouldn't describe it as "straight-forward" ;-)
                  • dharbari2
                    Would love to get a copy too. vijay@mindspring.com thanks Vijay
                    Message 9 of 9 , Jul 3, 2003
                    • 0 Attachment
                      Would love to get a copy too.
                      vijay@...
                      thanks

                      Vijay

                      --- In antlr-interest@yahoogroups.com, "atripp54321" <atripp@c...>
                      wrote:
                      > I have a straightforward JavaEmitter class that produces
                      > decent-looking java code from an ANTLR AST.
                      > I'll mail it to you (I assume it's poor edicate to post
                      > 900 lines of code).
                      >
                      > Andy
                      >
                      > --- In antlr-interest@yahoogroups.com, "Tim Tyler" <tt2333@y...>
                      > wrote:
                      > > I'm contemplating an ANTLR translation project.
                      > >
                      > > To do this I need to reconstruct the Java file from its tree.
                      > >
                      > > There seem to be several problems.
                      > >
                      > > One is that unicode information seems to get blatted.
                      > >
                      > > Another is that most of the Java parsers I can see
                      > > strip out comments and white space. I'm likely to
                      > > be wanting to preserve these as much as possible -
                      > > and maybe have them in the parse tree.
                      > >
                      > > About the only existing Java ANTLR translation project
                      > > I'm aware of is Jalopy: http://jalopy.sourceforge.net/
                      > >
                      > > Jalopy suggests that something is at least possible - but
                      > > it is big and complex - and I'd like a bit of a less hairy
                      > > example.
                      > >
                      > > Are there any other ANTLR translation projects out there?
                      > >
                      > > I'm a bit suprised ANTLR isn't already a little more translation-
                      > > oriented.
                      > >
                      > > After all, being able to reconstruct the original file is a good
                      > > test for having parsed it correctly ;-)
                    Your message has been successfully submitted and would be delivered to recipients shortly.