Loading ...
Sorry, an error occurred while loading the content.

multilexer

Expand Messages
  • craigmain001 <craig@palantir.co.za>
    Hi, I am using antlr 2.7.2 I have created two lexers, one that merely copies the input to output using a filter, and another that reads language statements
    Message 1 of 1 , Feb 5, 2003
    View Source
    • 0 Attachment
      Hi,

      I am using antlr 2.7.2

      I have created two lexers, one that merely copies the input to output
      using a filter, and another that reads language statements that are
      contained in delimited sections of the input.

      It almost works. I have shown the output below. I have two questions.

      Why does the parser recognise if before it recognises foreach?
      Why does the selector->pop not work. Why does it stop parsing at that
      point? How do I get the lexer to continue reading????

      Please help.
      Regards
      Craig.

      The test input looks as follows.
      ==========================
      this is a test
      int i = 0;
      procedure pop;
      [foreach]
      [if]
      end
      ==========================
      The output produces is as follows.
      ==========================
      this is a test
      int i = 0;
      procedure pop;
      {ifstatement encountered}
      ==========================

      Here are the files.
      xgen.g
      ==========================================================
      header {
      #include "antlr/TokenStreamSelector.hpp"
      #include <iostream>
      }

      options
      { language="Cpp"; }

      class XGenParser extends Parser;

      statement : (forstatement | ifstatement);


      ifstatement : IF
      { std::cout << "{ifstatement encountered}"; }
      ;

      forstatement : FOREACH
      { std::cout << "{foreach encountered}"; }
      ;

      class XGenLexer extends Lexer;
      options {
      k=2;
      }


      tokens {
      FOREACH="foreach";
      IF="if";
      }


      {
      private:
      ANTLR_USE_NAMESPACE(antlr)TokenStreamSelector* selector;
      public:
      void setSelector(ANTLR_USE_NAMESPACE(antlr)TokenStreamSelector*
      selector_) {
      selector=selector_;
      }
      }

      CODE_END
      : "]" {selector->pop(); }
      ;


      WS_ : (' '
      | '\t'
      | '\n'
      | '\r')
      { _ttype = ANTLR_USE_NAMESPACE(antlr)Token::SKIP; }
      ;

      ==========================================================
      code.g
      header {
      #include "antlr/TokenStreamSelector.hpp"
      #include <iostream>
      }

      options {
      language="Cpp";
      }

      class CodeLexer extends Lexer;
      options {
      k=2;
      filter=IGNORE;
      charVocabulary='\u0000'..'\u007F'; // ASCII
      }
      {
      private:
      ANTLR_USE_NAMESPACE(antlr)TokenStreamSelector* selector;
      public:
      void setSelector(ANTLR_USE_NAMESPACE(antlr)TokenStreamSelector*
      selector_) {
      selector=selector_;
      }
      }

      CODE_BEGIN : "[" {selector->push("xgen"); }
      ;

      protected
      IGNORE
      : ( "\r\n" /*| '\r' | '\n'*/ )
      {newline(); std::cout << std::endl;}
      | c:. {std::cout << c;}
      ;
      header {
      #include "antlr/TokenStreamSelector.hpp"
      #include <iostream>
      }

      options {
      language="Cpp";
      }

      class CodeLexer extends Lexer;
      options {
      k=2;
      filter=IGNORE;
      charVocabulary='\u0000'..'\u007F'; // ASCII
      }
      {
      private:
      ANTLR_USE_NAMESPACE(antlr)TokenStreamSelector* selector;
      public:
      void setSelector(ANTLR_USE_NAMESPACE(antlr)TokenStreamSelector*
      selector_) {
      selector=selector_;
      }
      }

      CODE_BEGIN : "[" {selector->push("xgen"); }
      ;

      protected
      IGNORE
      : ( "\r\n" /*| '\r' | '\n'*/ )
      {newline(); std::cout << std::endl;}
      | c:. {std::cout << c;}
      ;

      ==========================================================
    Your message has been successfully submitted and would be delivered to recipients shortly.