Loading ...
Sorry, an error occurred while loading the content.

Stuck in treewalkers

Expand Messages
  • Lars Clausen
    I am having the hardest time getting a treewalker to accept my ast. The parser works beautifully, and prints out the AST as it should be, but that treewalker
    Message 1 of 2 , Sep 30, 1998
    • 0 Attachment
      I am having the hardest time getting a treewalker to accept my ast. The
      parser works beautifully, and prints out the AST as it should be, but that
      treewalker won't accept anything, not even a simple integer. It's driving
      me crazy, as I can't see any difference between what I do here and what the
      docs say. It seems like the keywords (SEQ, NUMBER, "infixl" etc) aren't
      recognized by the walker at all. Please, what am I doing wrong?

      -Lars

      {// Generated from miniJR.g by antlr
      import java.util.*;

      }

      class miniParser extends Parser;
      options {
      k=2;
      tokenVocabulary=miniTokens;
      buildAST=true;
      }

      startRule
      : ( topExpression ";"! )*
      ;

      topExpression
      : expression
      | functiondecl
      | infixfunctiondecl
      ;

      sequence
      : hd:expression tl:( ";"! expression )*
      { ## = #(#[SEQ, "seq"], hd, tl); }
      ;

      expression
      : IDENT ":="^ nonrefexpression
      | nonrefexpression
      ;

      nonrefexpression
      : app ( OP^ app )*
      | "fn"^ identlist "=>"! nonrefexpression
      ;

      app
      : term ( term )*
      {
      if (##.getNextSibling() != null)
      ## = #(#[APP, "app"], ##);
      }
      ;

      term
      : "("! sequence ")"!
      | "("! "op"^ OP ")"!
      | triv
      | "let"^ ( letform )+ "in"! sequence "end"!
      ;

      letform
      : "val"^ IDENT "="! nonrefexpression
      ;

      triv
      : IDENT { ## = #(#[VAR, "var"], ##); }
      | INTEGER { ## = #(#[NUMBER, "NUMBER"], ##); }
      ;

      functiondecl
      : "fun"^ f:IDENT args:identlist "="! nonrefexpression
      ;

      identlist:
      ( IDENT )+
      { ## = #(#[PARAMS, "params"], ##); }
      ;

      infixfunctiondecl
      : "infixl"^ INTEGER ( IDENT | OP ) opfunctiondecl
      | "infixr"^ INTEGER ( IDENT | OP ) opfunctiondecl
      ;

      opfunctiondecl
      : "fun"^ name:( OP | IDENT ) identlist "="! nonrefexpression
      ;

      imaginaryTokens: APP | SEQ | VAR | PARAMS | NUMBER | LET | FUN | INFIXL | INFIXR ;

      class miniLexer extends Lexer;
      options {
      k=3;
      tokenVocabulary=miniTokens;
      literal="=";
      literal="=>";
      literal=";";
      literal="(";
      literal=")";
      literal="op";
      literal=":=";
      }

      IDENT
      : ( 'a'..'z' | 'A'..'Z' ) ( 'a'..'z' | 'A'..'Z' | '_' | '0'..'9' )*
      ;

      INTEGER
      : ( '0'..'9' ) ( '0'..'9' )*
      ;

      SEMI : ';' ;
      OPEN : '(' ;
      CLOSE : ')' ;

      OP
      :
      ( '<' | '>' | '=' | '-' | '@' | '#' | '!' | '$' | '^' | ":" |
      '&' | '*' | '_' | '+' | '|' | '?' | '.' | ',' | '~' )+
      ;

      COMMENT
      : '/' '/' ( ~ ( '\n' | '\r' ) )* ( '\n' | '\r' '\n' ) { newline(); }
      { $setType(Token.SKIP); } // ignore this token
      ;

      WS
      : ( ' '
      | '\t'
      | '\r' '\n' { newline(); }
      | '\n' { newline(); }
      )
      { $setType(Token.SKIP); } // ignore this token
      ;

      class miniJRWalker extends TreeParser;

      term
      { System.out.println("Walking..."); }
      : #( VAR IDENT )
      | #( OP term term )
      | { System.out.println("Let's call it a number"); }
      #( NUMBER INTEGER ) { System.out.println("4"); }
      | #( "app" IDENT (IDENT)* )
      | #( SEQ ( term )+ )
      | #( LET letvals term )
      | #( FUN IDENT ( PARAMS ) term )
      | #( "infixl" NUMBER (IDENT|OP) ( PARAMS ) term )
      | #( INFIXR NUMBER (IDENT|OP) ( PARAMS ) term )
      | #( ":=" IDENT term)
      ;

      number
      : INTEGER
      ;

      letvals:
      : #( LET IDENT term )+
      ;

      --
      Lars R. Clausen (lrclause@...)
      A *real* smart bomb would call in sick, perhaps move to another country,
      changing its name in the process, open a beach bar maybe and live out its
      days in safe anonymity. -- Barry O'Neill in rhod
    • Monty Zukowski
      ... From: Lars Clausen To: antlr-interest@onelist.com Date: Wednesday, September 30, 1998 8:01 PM
      Message 2 of 2 , Oct 1, 1998
      • 0 Attachment
        -----Original Message-----
        From: Lars Clausen <lrclause@...>
        To: antlr-interest@onelist.com <antlr-interest@onelist.com>
        Date: Wednesday, September 30, 1998 8:01 PM
        Subject: [antlr-interest] Stuck in treewalkers


        >From: Lars Clausen <lrclause@...>
        >
        >
        >I am having the hardest time getting a treewalker to accept my ast. The
        >parser works beautifully, and prints out the AST as it should be, but that
        >treewalker won't accept anything, not even a simple integer. It's driving
        >me crazy, as I can't see any difference between what I do here and what the
        >docs say. It seems like the keywords (SEQ, NUMBER, "infixl" etc) aren't
        >recognized by the walker at all. Please, what am I doing wrong?
        >


        You need to specify the tokenVocabulary for your TreeParser too!

        class miniJRWalker extends TreeParser;
        options

        tokenVocabulary=miniTokens;
        }

        When you get into multiple tree parsers and start using the tokdef option,
        you will undoubtedly change your grammar which will change the token types
        file. Then if you don't rebuild all of your tree parsers you will have
        errors where the token type doesn't match what the parser had. All of a
        sudden the INT from the parser is now the STRING in the tree walker, etc.
        It may look like your parser is building wacky trees, but it's just that the
        token types no longer match. Rebuild all your tree parsers and you'll be
        ok.

        Monty
      Your message has been successfully submitted and would be delivered to recipients shortly.