Loading ...
Sorry, an error occurred while loading the content.

importVocab exportVocab problem

Expand Messages
  • hawkwall
    Hello, I am having trouble defining a vocabulary in one file and the importing it into my Parser. I have read the documentation and everything I can find in
    Message 1 of 4 , Jan 3, 2004
    • 0 Attachment
      Hello,

      I am having trouble defining a vocabulary in one file and the
      importing it into my Parser. I have read the documentation and
      everything I can find in the newgroup, plus have gone through the
      TinyBasic example and I am still getting errors. Sorry, this seems
      like it should be easy.

      I started by defining a lexer and parser in the same file. When I did
      this, my example ran. I then created another grammar file, moved my
      lexer to this file, added an exportVocab option. In the original
      file, I deleted the lexer and added an importVocab option to the
      parser. Compiling the lexer first and then the parser, I see the
      tokens are generated in the ParserTokensTypes.txt, but I get the
      following error:

      line 1:1: expecting "MIKE.START", found 'MIKE.START'

      Here is the Lexer:
      class mikeLexer extends Lexer;

      options {
      testLiterals=true;
      exportVocab=mike;
      }

      protected
      DIGIT : ('0'..'9') ;

      protected
      DOT : '.' ;

      protected
      INTEGER : (DIGIT)+ ;


      protected
      DECIMAL : (DIGIT)* DOT (DIGIT)+ ;

      NUMBER : ((DIGIT)* DOT)=> (DIGIT)* DOT (DIGIT)+
      {$setType(DECIMAL);}
      | (DIGIT)+
      {$setType(INTEGER);}
      ;

      protected
      LETTER : 'A'..'Z';

      TAG : (LETTER | DOT)+ ;


      WS : ( ' '
      | '\t'
      | '\f'
      | ( options {generateAmbigWarnings=false;}
      : "\r\n" // DOS
      | '\r' // Macintosh
      | '\n' // Unix
      )
      {newline();}
      )+

      // now the overall whitespace action -- skip it!
      { $setType(Token.SKIP); }
      ;

      Here is the parser:
      class P extends Parser;

      options {
      k=2;
      importVocab=mike;
      }

      startRule : "MIKE.START"
      INTEGER
      "END.MIKE"
      ;


      Here is the input file:
      MIKE.START
      100
      END.MIKE

      My goal is be able to share this Vocabulary with multiple Parsers, at
      times extending the tokens. What am I missing? Thanks for your time.

      Mike Wall
    • Terence Parr
      are you running antlr on the lexer first (the output vocab) and then on the grammar (that uses the exported vocab)? Ter ... -- Professor Comp. Sci., University
      Message 2 of 4 , Jan 3, 2004
      • 0 Attachment
        are you running antlr on the lexer first (the output vocab) and then on
        the grammar (that uses the exported vocab)?

        Ter
        On Saturday, January 3, 2004, at 05:47 AM, hawkwall wrote:

        > Hello,
        >
        > I am having trouble defining a vocabulary in one file and the
        > importing it into my Parser. I have read the documentation and
        > everything I can find in the newgroup, plus have gone through the
        > TinyBasic example and I am still getting errors. Sorry, this seems
        > like it should be easy.
        >
        > I started by defining a lexer and parser in the same file. When I did
        > this, my example ran. I then created another grammar file, moved my
        > lexer to this file, added an exportVocab option. In the original
        > file, I deleted the lexer and added an importVocab option to the
        > parser. Compiling the lexer first and then the parser, I see the
        > tokens are generated in the ParserTokensTypes.txt, but I get the
        > following error:
        >
        > line 1:1: expecting "MIKE.START", found 'MIKE.START'
        >
        > Here is the Lexer:
        > class mikeLexer extends Lexer;
        >
        > options {
        > testLiterals=true;
        > exportVocab=mike;
        > }
        >
        > protected
        > DIGIT : ('0'..'9') ;
        >
        > protected
        > DOT : '.' ;
        >
        > protected
        > INTEGER : (DIGIT)+ ;
        >
        >
        > protected
        > DECIMAL : (DIGIT)* DOT (DIGIT)+ ;
        >
        > NUMBER : ((DIGIT)* DOT)=> (DIGIT)* DOT (DIGIT)+
        > {$setType(DECIMAL);}
        > | (DIGIT)+
        > {$setType(INTEGER);}
        > ;
        >
        > protected
        > LETTER : 'A'..'Z';
        >
        > TAG : (LETTER | DOT)+ ;
        >
        >
        > WS : ( ' '
        > | '\t'
        > | '\f'
        > | ( options {generateAmbigWarnings=false;}
        > : "\r\n" // DOS
        > | '\r' // Macintosh
        > | '\n' // Unix
        > )
        > {newline();}
        > )+
        >
        > // now the overall whitespace action -- skip it!
        > { $setType(Token.SKIP); }
        > ;
        >
        > Here is the parser:
        > class P extends Parser;
        >
        > options {
        > k=2;
        > importVocab=mike;
        > }
        >
        > startRule : "MIKE.START"
        > INTEGER
        > "END.MIKE"
        > ;
        >
        >
        > Here is the input file:
        > MIKE.START
        > 100
        > END.MIKE
        >
        > My goal is be able to share this Vocabulary with multiple Parsers, at
        > times extending the tokens. What am I missing? Thanks for your time.
        >
        > Mike Wall
        >
        >
        >
        >
        > Yahoo! Groups Links
        >
        > To visit your group on the web, go to:
        > http://groups.yahoo.com/group/antlr-interest/
        >
        > To unsubscribe from this group, send an email to:
        > antlr-interest-unsubscribe@yahoogroups.com
        >
        > Your use of Yahoo! Groups is subject to:
        > http://docs.yahoo.com/info/terms/
        >
        >
        >
        --
        Professor Comp. Sci., University of San Francisco
        Creator, ANTLR Parser Generator, http://www.antlr.org
        Co-founder, http://www.jguru.com
        Co-founder, http://www.knowspam.net enjoy email again!
        Co-founder, http://www.peerscope.com link sharing, pure-n-simple
      • hawkwall
        Thanks for the reply, Yes, I think so. I cleared out all generated java and txt files, ran antlr.Tool on the lexer and then on the parser which contains the
        Message 3 of 4 , Jan 3, 2004
        • 0 Attachment
          Thanks for the reply,

          Yes, I think so. I cleared out all generated java and txt files, ran
          antlr.Tool on the lexer and then on the parser which contains the
          grammar. Seems like it should work, and it looks like the grammar is
          importing the vocab correctly. I there something in the way antlr
          uses a lexer and a parser in the same file that I need to do when I
          break them into separate files? I appreciate the reply.

          Mike


          --- In antlr-interest@yahoogroups.com, Terence Parr <parrt@c...> wrote:
          > are you running antlr on the lexer first (the output vocab) and then on
          > the grammar (that uses the exported vocab)?
          >
          > Ter
          > On Saturday, January 3, 2004, at 05:47 AM, hawkwall wrote:
          >
          > > Hello,
          > >
          > > I am having trouble defining a vocabulary in one file and the
          > > importing it into my Parser. I have read the documentation and
          > > everything I can find in the newgroup, plus have gone through the
          > > TinyBasic example and I am still getting errors. Sorry, this seems
          > > like it should be easy.
          > >
          > > I started by defining a lexer and parser in the same file. When I did
          > > this, my example ran. I then created another grammar file, moved my
          > > lexer to this file, added an exportVocab option. In the original
          > > file, I deleted the lexer and added an importVocab option to the
          > > parser. Compiling the lexer first and then the parser, I see the
          > > tokens are generated in the ParserTokensTypes.txt, but I get the
          > > following error:
          > >
          > > line 1:1: expecting "MIKE.START", found 'MIKE.START'
          > >
          > > Here is the Lexer:
          > > class mikeLexer extends Lexer;
          > >
          > > options {
          > > testLiterals=true;
          > > exportVocab=mike;
          > > }
          > >
          > > protected
          > > DIGIT : ('0'..'9') ;
          > >
          > > protected
          > > DOT : '.' ;
          > >
          > > protected
          > > INTEGER : (DIGIT)+ ;
          > >
          > >
          > > protected
          > > DECIMAL : (DIGIT)* DOT (DIGIT)+ ;
          > >
          > > NUMBER : ((DIGIT)* DOT)=> (DIGIT)* DOT (DIGIT)+
          > > {$setType(DECIMAL);}
          > > | (DIGIT)+
          > > {$setType(INTEGER);}
          > > ;
          > >
          > > protected
          > > LETTER : 'A'..'Z';
          > >
          > > TAG : (LETTER | DOT)+ ;
          > >
          > >
          > > WS : ( ' '
          > > | '\t'
          > > | '\f'
          > > | ( options {generateAmbigWarnings=false;}
          > > : "\r\n" // DOS
          > > | '\r' // Macintosh
          > > | '\n' // Unix
          > > )
          > > {newline();}
          > > )+
          > >
          > > // now the overall whitespace action -- skip it!
          > > { $setType(Token.SKIP); }
          > > ;
          > >
          > > Here is the parser:
          > > class P extends Parser;
          > >
          > > options {
          > > k=2;
          > > importVocab=mike;
          > > }
          > >
          > > startRule : "MIKE.START"
          > > INTEGER
          > > "END.MIKE"
          > > ;
          > >
          > >
          > > Here is the input file:
          > > MIKE.START
          > > 100
          > > END.MIKE
          > >
          > > My goal is be able to share this Vocabulary with multiple Parsers, at
          > > times extending the tokens. What am I missing? Thanks for your time.
          > >
          > > Mike Wall
          > >
          > >
          > >
          > >
          > > Yahoo! Groups Links
          > >
          > > To visit your group on the web, go to:
          > > http://groups.yahoo.com/group/antlr-interest/
          > >
          > > To unsubscribe from this group, send an email to:
          > > antlr-interest-unsubscribe@yahoogroups.com
          > >
          > > Your use of Yahoo! Groups is subject to:
          > > http://docs.yahoo.com/info/terms/
          > >
          > >
          > >
          > --
          > Professor Comp. Sci., University of San Francisco
          > Creator, ANTLR Parser Generator, http://www.antlr.org
          > Co-founder, http://www.jguru.com
          > Co-founder, http://www.knowspam.net enjoy email again!
          > Co-founder, http://www.peerscope.com link sharing, pure-n-simple
        • Terence Parr
          hmm...yep, looks like a literals issue. YOu define a literal in the parser, but it cannot be fed back into the lexer when the lexer exports the vocab. It
          Message 4 of 4 , Jan 3, 2004
          • 0 Attachment
            hmm...yep, looks like a literals issue. YOu define a literal in the
            parser, but it cannot be fed back into the lexer when the lexer exports
            the vocab. It returns as TAG not "MIKE.START". Define the literals in
            the lexer I'd say and then you can use in the parser. :)

            Ter
            On Saturday, January 3, 2004, at 03:45 PM, hawkwall wrote:

            > Thanks for the reply,
            >
            > Yes, I think so. I cleared out all generated java and txt files, ran
            > antlr.Tool on the lexer and then on the parser which contains the
            > grammar. Seems like it should work, and it looks like the grammar is
            > importing the vocab correctly. I there something in the way antlr
            > uses a lexer and a parser in the same file that I need to do when I
            > break them into separate files? I appreciate the reply.
            >
            > Mike
            >
            >
            > --- In antlr-interest@yahoogroups.com, Terence Parr <parrt@c...> wrote:
            >> are you running antlr on the lexer first (the output vocab) and then
            >> on
            >> the grammar (that uses the exported vocab)?
            >>
            >> Ter
            >> On Saturday, January 3, 2004, at 05:47 AM, hawkwall wrote:
            >>
            >>> Hello,
            >>>
            >>> I am having trouble defining a vocabulary in one file and the
            >>> importing it into my Parser. I have read the documentation and
            >>> everything I can find in the newgroup, plus have gone through the
            >>> TinyBasic example and I am still getting errors. Sorry, this seems
            >>> like it should be easy.
            >>>
            >>> I started by defining a lexer and parser in the same file. When I
            >>> did
            >>> this, my example ran. I then created another grammar file, moved my
            >>> lexer to this file, added an exportVocab option. In the original
            >>> file, I deleted the lexer and added an importVocab option to the
            >>> parser. Compiling the lexer first and then the parser, I see the
            >>> tokens are generated in the ParserTokensTypes.txt, but I get the
            >>> following error:
            >>>
            >>> line 1:1: expecting "MIKE.START", found 'MIKE.START'
            >>>
            >>> Here is the Lexer:
            >>> class mikeLexer extends Lexer;
            >>>
            >>> options {
            >>> testLiterals=true;
            >>> exportVocab=mike;
            >>> }
            >>>
            >>> protected
            >>> DIGIT : ('0'..'9') ;
            >>>
            >>> protected
            >>> DOT : '.' ;
            >>>
            >>> protected
            >>> INTEGER : (DIGIT)+ ;
            >>>
            >>>
            >>> protected
            >>> DECIMAL : (DIGIT)* DOT (DIGIT)+ ;
            >>>
            >>> NUMBER : ((DIGIT)* DOT)=> (DIGIT)* DOT (DIGIT)+
            >>> {$setType(DECIMAL);}
            >>> | (DIGIT)+
            >>> {$setType(INTEGER);}
            >>> ;
            >>>
            >>> protected
            >>> LETTER : 'A'..'Z';
            >>>
            >>> TAG : (LETTER | DOT)+ ;
            >>>
            >>>
            >>> WS : ( ' '
            >>> | '\t'
            >>> | '\f'
            >>> | ( options {generateAmbigWarnings=false;}
            >>> : "\r\n" // DOS
            >>> | '\r' // Macintosh
            >>> | '\n' // Unix
            >>> )
            >>> {newline();}
            >>> )+
            >>>
            >>> // now the overall whitespace action -- skip it!
            >>> { $setType(Token.SKIP); }
            >>> ;
            >>>
            >>> Here is the parser:
            >>> class P extends Parser;
            >>>
            >>> options {
            >>> k=2;
            >>> importVocab=mike;
            >>> }
            >>>
            >>> startRule : "MIKE.START"
            >>> INTEGER
            >>> "END.MIKE"
            >>> ;
            >>>
            >>>
            >>> Here is the input file:
            >>> MIKE.START
            >>> 100
            >>> END.MIKE
            >>>
            >>> My goal is be able to share this Vocabulary with multiple Parsers, at
            >>> times extending the tokens. What am I missing? Thanks for your time.
            >>>
            >>> Mike Wall
            >>>
            >>>
            >>>
            >>>
            >>> Yahoo! Groups Links
            >>>
            >>> To visit your group on the web, go to:
            >>> http://groups.yahoo.com/group/antlr-interest/
            >>>
            >>> To unsubscribe from this group, send an email to:
            >>> antlr-interest-unsubscribe@yahoogroups.com
            >>>
            >>> Your use of Yahoo! Groups is subject to:
            >>> http://docs.yahoo.com/info/terms/
            >>>
            >>>
            >>>
            >> --
            >> Professor Comp. Sci., University of San Francisco
            >> Creator, ANTLR Parser Generator, http://www.antlr.org
            >> Co-founder, http://www.jguru.com
            >> Co-founder, http://www.knowspam.net enjoy email again!
            >> Co-founder, http://www.peerscope.com link sharing, pure-n-simple
            >
            >
            >
            >
            > Yahoo! Groups Links
            >
            > To visit your group on the web, go to:
            > http://groups.yahoo.com/group/antlr-interest/
            >
            > To unsubscribe from this group, send an email to:
            > antlr-interest-unsubscribe@yahoogroups.com
            >
            > Your use of Yahoo! Groups is subject to:
            > http://docs.yahoo.com/info/terms/
            >
            >
            >
            --
            Professor Comp. Sci., University of San Francisco
            Creator, ANTLR Parser Generator, http://www.antlr.org
            Co-founder, http://www.jguru.com
            Co-founder, http://www.knowspam.net enjoy email again!
            Co-founder, http://www.peerscope.com link sharing, pure-n-simple
          Your message has been successfully submitted and would be delivered to recipients shortly.