Loading ...
Sorry, an error occurred while loading the content.

submitted AST factory changes to repository

Expand Messages
  • Terence Parr
    Folks, I think I have fixed up the Java heterogeneous tree stuff and wonder of all wondes, I have added a TestASTFactory class in the examples area! ... I have
    Message 1 of 3 , Nov 29, 2002
    • 0 Attachment
      Folks,

      I think I have fixed up the Java heterogeneous tree stuff and wonder of
      all wondes, I have added a TestASTFactory class in the examples area!
      My previous email + comments:

      > 1. #[FOO] always builds an AST node of the default type because
      > the ASTFactory only knows about the default. In future if you say
      >
      > tokens {
      > PLUS<AST=PLUSNode>;
      > ...
      > }
      > then I'll make action #[PLUS] create the right node. You can also
      > say
      > #[ID,"foo","VarNode"] (3rd arg is the type of node to create).

      I have modified action.g to handle the 3rd arg.

      For 1 and 2 arg #[...] constructors, JavaCodeGenerator has been
      modified to handle snooping the first argument, ID in this case, and
      asking for any specified AST node type. If a node type is known, it is
      passed as a third arg to create(...).

      > 2. dup methods of ASTFactory don't respect the type of the nodes; it
      > uses default node type. In future, i'll use
      > t.getClass().newInstance()
      > to do the dup.

      ASTFactory.dup(AST t) now makes sure that the resulting node has the
      same type, t.getClass(), as t.

      > 3. hetero tree construction does not call the factory. E.g.,
      >
      > anIntRule : INT<AST=INTNode> ;
      >
      > generates
      >
      > INTNode v = new INTNode(LT(1));
      >
      > but we need to instead generate:
      >
      > AST v = (AST)astFactory.create(LT(1),"INTNode");
      >
      > where the create(...) method is new and specifies the type to
      > create. This will use newInstance() instead of "new" by default.

      Every reference to INT now generates

      INTNode tmp1_AST = null;
      tmp1_AST = (INTNode)astFactory.create(LT(1),"INTNode");

      not

      INTNode tmp1_AST = null;
      tmp1_AST = new INTNode(LT(1));

      and so does stuff like X<AST=XNode>.

      > 4. If you define ID<AST=T> in tokens section then all code in grammar
      > "id:ID" should
      > define labels as "T id" not "AST id" nor labelASTType id.

      I think it was this way before, but ANTLR definitely uses the most
      specific type it can now.

      tokens {
      A<AST=X>;
      }

      a : A A<AST=Y> ;

      generates

      X tmp1_AST = null;
      tmp1_AST = (X)astFactory.create(LT(1),"X");
      ...
      Y tmp2_AST = null;
      tmp2_AST = (Y)astFactory.create(LT(1),"Y");

      I have modified the following files:
      ----------------------------------------------

      //depot/code/org.antlr/main/main/antlr/ASTFactory.java # edit

      Added mapping from token type to classname for dynamic node
      specification.
      ANTLR doesn't use this as it only handles static stuff like #[ID] not
      #[LT(1)].

      public void setTokenTypeASTNodeType(int tokenType, String
      className)

      modified all of the create/dup routines to respect type. Added

      public AST create(int type, String txt, String className)

      to handle #[ID,"v","MyNodeType"].

      //depot/code/org.antlr/main/main/antlr/BaseAST.java # edit

      Just edited comment.

      //depot/code/org.antlr/main/main/antlr/JavaCodeGenerator.java
      # edit

      Altered the getASTCreateString(...) routines to check for AST node type
      stuff

      //depot/code/org.antlr/main/main/antlr/Parser.java # edit

      Changed a comment

      //depot/code/org.antlr/main/main/antlr/Version.java # edit

      //depot/code/org.antlr/main/main/antlr/actions/java/ActionLexer.java
      # edit

      //depot/code/org.antlr/main/main/antlr/actions/java/
      ActionLexerTokenTypes.java # edit
      //depot/code/org.antlr/main/main/antlr/actions/java/action.g
      # edit

      Added #[a,b,c] format


      //depot/code/org.antlr/main/main/examples/java/ASTsupport/
      ASTType49.java # add

      //depot/code/org.antlr/main/main/examples/java/ASTsupport/MyAST.java
      # add

      //depot/code/org.antlr/main/main/examples/java/ASTsupport/
      TestASTFactory.java # add

      unit test


      //depot/code/org.antlr/main/main/examples/java/heteroAST/
      BinaryOperatorAST.java # edit

      //depot/code/org.antlr/main/main/examples/java/heteroAST/CalcAST.java
      # edit

      //depot/code/org.antlr/main/main/examples/java/heteroAST/INTNode.java
      # edit

      //depot/code/org.antlr/main/main/examples/java/heteroAST/MULTNode.java
      # edit

      //depot/code/org.antlr/main/main/examples/java/heteroAST/Main.java
      # edit

      //depot/code/org.antlr/main/main/examples/java/heteroAST/PLUSNode.java
      # edit

      Factored some init methods out of specific AST types.

      I have uploaded this antlr-2.7.2a6_11-29-2002.jar file to the download
      area if you want to check it out on your grammars. I think C++, C#,
      Java are now more similar in their treatment of hetero trees; for sure,
      the java hetero tree stuff was just plain broken before.

      Next step is C# integration into the main line.

      Ter
      --
      Co-founder, http://www.jguru.com
      Creator, ANTLR Parser Generator: http://www.antlr.org
      Lecturer in Comp. Sci., University of San Francisco
    • Ric Klaren
      Hi, ... A bit late in speaking up about this.. but is this really something you want to do/support? It allows a user to invalidate the mapping of a type number
      Message 2 of 3 , Dec 2, 2002
      • 0 Attachment
        Hi,

        On Fri, Nov 29, 2002 at 03:40:06PM -0800, Terence Parr wrote:
        > > 1. #[FOO] always builds an AST node of the default type because
        > > the ASTFactory only knows about the default. In future if you say
        > >
        > > tokens {
        > > PLUS<AST=PLUSNode>;
        > > ...
        > > }
        > > then I'll make action #[PLUS] create the right node. You can also
        > > say
        > > #[ID,"foo","VarNode"] (3rd arg is the type of node to create).
        >
        > I have modified action.g to handle the 3rd arg.

        A bit late in speaking up about this.. but is this really something you
        want to do/support? It allows a user to invalidate the mapping of a type
        number to a instance of a certain AST node class. It sounds a bit like
        providing a user with quite some rope.. or a nice new shiny handgun to hit
        his/her foot :)

        If a user makes some code to do stuff on an AST which checks the type
        numbers and not the exact class type he may get weird results if you
        support this. I'd at least tag it with a big 'know-what-your-are-doing' tag
        in the documentation.

        I cannot really think of a case where you'd even want to do this... But as
        usual feel free to correct me =)

        Cheers,

        Ric
        --
        -----+++++*****************************************************+++++++++-------
        ---- Ric Klaren ----- klaren@... ----- +31 53 4893722 ----
        -----+++++*****************************************************+++++++++-------
        "You know how to use that thing?" [pointing to the sword]
        "Sure.. The pointy end goes into the other guy."
        --- The Mask of Zorro
      • Ric Klaren
        Hi, ... I don t see your last point actually. I m all in favour of abstracting the AST construction/manipulation away from the target language. But the thing
        Message 3 of 3 , Dec 5, 2002
        • 0 Attachment
          Hi,

          On Mon, Dec 02, 2002 at 05:04:35PM -0800, Loring Craymer wrote:
          > You're probably right that this is not a needed construct as long as the
          > AST Factory does the right thing about mapping token types to AST
          > classes. However, I'd much rather see AST construction and manipulation
          > abstracted from the target language as much as possible to support better
          > optimization of generated code and easier targeting of multiple output
          > languages. This is a step in that direction.

          I don't see your last point actually. I'm all in favour of abstracting the
          AST construction/manipulation away from the target language. But the thing
          is that I don't see how a way to circumvent the mapping from the token type
          to the AST class instance pointed to will help in abstracting things (I
          might even tend to say obfuscate).

          If it's just having a complete as possible interface in the AST factory
          then I'd say no problem. In other cases I'm far from convinced that there
          is any merit to this. Okay this may just be a difference in opinion in
          coding 'style'. So if you guys think it's no problem just go ahead ;)

          > It would probably be a good idea to generate a warning when the assigned
          > token mapping is overridden--I suspect that most overrides would be
          > associated with a token type change, anyway.

          Am I missing something here? Are talking Lexer or Parser/Treewalker here?
          In the case of lexer I'd rather opt to support heterogenous tokens in stead
          of another 'hack' (it kindoff feels like the hardcoded 'new SomeASTType();'
          calls we just got rid off).

          > I think that we should put some thought into ANTLR 3 support for
          > user-defined attributes (get/set ops, for example; automatic AST class
          > construction, for another). Attributes are usually not specific to a
          > target language and might as well be supported directly.

          I wish I had more time for discussions on these things and on working on
          antlr :(

          Cheers,

          Ric
          --
          -----+++++*****************************************************+++++++++-------
          ---- Ric Klaren ----- klaren@... ----- +31 53 4893722 ----
          -----+++++*****************************************************+++++++++-------
          "You can't expect to wield supreme executive power just because some
          watery tot throws a sword at you!"
          --- Monty Python and the Holy Grail
        Your message has been successfully submitted and would be delivered to recipients shortly.