Loading ...
Sorry, an error occurred while loading the content.

Context-sensitive grammar

Expand Messages
  • Jim Douglas
    I ve just started to review ANTLR, and my first question is pretty basic: Is ANTLR a good tool for implementing a grammar for an extremely context-sensitive
    Message 1 of 3 , Feb 2, 1999
    • 0 Attachment
      I've just started to review ANTLR, and my first question is pretty basic: Is
      ANTLR a good tool for implementing a grammar for an extremely
      context-sensitive language? Here's an example of a legal statement:

      IF THEN THEN THEN=ELSE ELSE ELSE=THEN

      Which means:

      VERB(IF), NUMEXPR("THEN"), VERB(THEN), NUMEXPR("THEN"), EQUALS,
      NUMEXPR("ELSE"), VERB(ELSE), NUMEXPR("ELSE"), EQUALS, NUMEXPR("THEN").

      Most sample grammars that I've seen assume that verbs are all reserved words;
      unfortunately, that's not an option for me.

      Jim.
    • Mcree, Randy
      Jim, I m not sure that ANTLR is the best tool, but you can do this. I had the same problem in parsing OMG s new grammar. They added keywords in context . So,
      Message 2 of 3 , Feb 2, 1999
      • 0 Attachment
        Jim,
        I'm not sure that ANTLR is the best tool, but you can do this. I had the
        same problem in parsing OMG's new grammar. They added "keywords in context".
        So, for example, "ValueBase" may be either a keyword or an identifier
        depending on context. Here's (parts of) my solution:
        grammar snippet:
        simple_type_spec
        : (IDENT) => { matches(LT(1).getText(), "ValueBase") }?
        ( value_type_spec_token )
        | base_type_spec
        | template_type_spec
        | scoped_name
        ;

        The lexer always returns these as IDENT so whenever they might appear I need
        to check for them and if so, tell ANTLR to take that branch. Once you get
        use to this it works and its not hard.

        You may run into ambiguities because of the wide use of IDENT. So to handle
        a case like that I have:
        // Isolate the ambiguity of an IDENT in the following context:
        // "value" IDENT. What follows could be
        // "supports" or a type specifier (a value box construct!)
        // other possibilities are ":", "{" or nothing (a forward
        declaration).
        //
        value_follower
        : (IDENT) => { matches(LT(1).getText(), "supports") }?
        ( support_specs LCURLY value_body RCURLY )
        | type_spec /* value_box_dcl */
        | ( value_inheritance_spec )? LCURLY value_body RCURLY
        ;
        In summary, I started with a grammar which did not use the ANTLR "=>"
        feature at all. I ended up with one that uses it extensively. You probably
        want to read up on syntactic and semantic predicates.

        Hope this helps.

        Randy McRee
        COMPAQ
        > -----Original Message-----
        > From: Jim Douglas [SMTP:jdouglas@...]
        > Sent: Tuesday, February 02, 1999 9:43 AM
        > To: antlr-interest@onelist.com
        > Subject: [antlr-interest] Context-sensitive grammar
        >
        > From: "Jim Douglas" <jdouglas@...>
        >
        > I've just started to review ANTLR, and my first question is pretty basic:
        > Is
        > ANTLR a good tool for implementing a grammar for an extremely
        > context-sensitive language? Here's an example of a legal statement:
        >
        > IF THEN THEN THEN=ELSE ELSE ELSE=THEN
        >
        > Which means:
        >
        > VERB(IF), NUMEXPR("THEN"), VERB(THEN), NUMEXPR("THEN"), EQUALS,
        > NUMEXPR("ELSE"), VERB(ELSE), NUMEXPR("ELSE"), EQUALS, NUMEXPR("THEN").
        >
        > Most sample grammars that I've seen assume that verbs are all reserved
        > words;
        > unfortunately, that's not an option for me.
        >
        > Jim.
        >
        >
        >
        > ------------------------------------------------------------------------
        > To unsubscribe from this mailing list, or to change your subscription
        > to digest, go to the ONElist web site, at http://www.onelist.com and
        > select the User Center link from the menu bar on the left.
      • pete@xxxxxx.xxxxx.xx.xxxxxxxxxxxxxx)
        ... It s certainly possible to do this in a number of ways. One way is to redefine what an identifier is in the parser, to include both general identifiers
        Message 3 of 3 , Feb 2, 1999
        • 0 Attachment
          >
          > From: "Jim Douglas" <jdouglas@...>
          >
          > I've just started to review ANTLR, and my first question is pretty basic: Is
          > ANTLR a good tool for implementing a grammar for an extremely
          > context-sensitive language? Here's an example of a legal statement:
          >
          > IF THEN THEN THEN=ELSE ELSE ELSE=THEN
          >
          > Which means:
          >
          > VERB(IF), NUMEXPR("THEN"), VERB(THEN), NUMEXPR("THEN"), EQUALS,
          > NUMEXPR("ELSE"), VERB(ELSE), NUMEXPR("ELSE"), EQUALS, NUMEXPR("THEN").
          >
          > Most sample grammars that I've seen assume that verbs are all reserved words;
          > unfortunately, that's not an option for me.

          It's certainly possible to do this in a number of ways. One way is to
          redefine what an identifier is in the parser, to include both general
          identifiers from the lexer and also the keywords of the language, eg:

          identifier
          : IDENT
          | "if"
          | "then"
          | "else"
          | ...
          ;

          Then, you can use this in your grammar, in place of the lexer's
          identifier. You will generate more ambiguities this way, but they should
          be easy to resolve, using syntactic predicates, eg:

          statement
          : ("if" expr)=>
          "if" expr "then" statement
          ( "else" statement )?
          | identifier EQUALS expr
          | ...
          ;

          Pete
        Your message has been successfully submitted and would be delivered to recipients shortly.