Loading ...
Sorry, an error occurred while loading the content.

Problems with " and/or ""

Expand Messages
  • matthew ford
    Attached is a corrected version of a lexer that will handle strings delimited by or " (actionLexer.g) Also attached is an alternate version which looks
    Message 1 of 3 , Dec 14, 2004
    • 0 Attachment
      Attached is a corrected version of a lexer that will handle strings
      delimited by " or " (actionLexer.g)
      Also attached is an alternate version which looks like it should work but
      does not due to the way Antlr generates that code. (badactionLexer.g)

      the interesting part of the working lexer is
      | ~('"'|'\n'|'\r')
      { if( ( (LA(0) == '&') && (LA(1) == 'q') && (LA(2) == 'u') &&
      (LA(3) == 'o') && (LA(4) == 't') && (LA(5) == ';')) ) {
      text.setLength(text.length()-1); // drop the & just matched
      break;
      } }
      )*
      ('"' !
      | ( 'q' ! 'u' ! 'o' ! 't' ! ';' !) // & already matched above
      )

      There should be an easier (and more strightforward) way to do this.
      But if there is I have not found it yet.
      Ter can you get greedy = false to work for this case?
      matthew
    • Terence Parr
      ... The unfortunate truth is that greedy=false was an afterthought and apparently not properly integrated. You ll note in 3.0 that this plain works: Q :
      Message 2 of 3 , Dec 15, 2004
      • 0 Attachment
        On Dec 14, 2004, at 11:16 PM, matthew ford wrote:
        > There should be an easier (and more strightforward) way to do this.
        > But if there is I have not found it yet.
        > Ter can you get greedy = false to work for this case?

        The unfortunate truth is that greedy=false was an afterthought and
        apparently not properly integrated.

        You'll note in 3.0 that this plain works:

        Q : """ (greedy=false : . )* """ ;

        I just verified the generated Java (no bytecodes in this case) and the
        DFA. Everything is cool. It will terminate on """ and more
        importantly, it figured out the k=6 lookahead depth out all by itself.
        :) Hooray. Believe me when I say that I've actually gotten time to
        think before coding this time around. 1.x was a prototype. 2.x was
        done in a hurry to shut out JavaCC and was done while I was in a
        start-up working like mad. I've been working on just the parsing
        algorithm and code generator for 1.5 years now. :) I should have the
        error handling done in a month or two. Then I clean up the tool and by
        May we may have a parser/lexer combo that works with no support for
        trees etc... We may also have a prototype gui.

        Ter
        --
        CS Professor & Grad Director, University of San Francisco
        Creator, ANTLR Parser Generator, http://www.antlr.org
        Cofounder, http://www.jguru.com
        Cofounder, http://www.knowspam.net enjoy email again!
      • matthew ford
        Sounds great Ter, just to clarify does ( | " | &) (greedy=false : . )* ( | " ) ; work also and also does ( | " | “ |
        Message 3 of 3 , Dec 15, 2004
        • 0 Attachment
          Sounds great Ter,
          just to clarify does

          ('"' | """ | "&) (greedy=false : . )* ('"' | """) ;

          work also and also does

          ('"' | """ | "“" | "”" ) (greedy=false : . )* ('"' ! |
          """ ! | "“" ! | "”" ! ) ;

          work?
          Looking forward to 3.0 !!! :-)
          matthew

          ----- Original Message -----
          From: "Terence Parr" <parrt@...>
          To: <antlr-interest@yahoogroups.com>
          Sent: Thursday, December 16, 2004 4:51 AM
          Subject: Re: [antlr-interest] Problems with " and/or ""


          >
          >
          > On Dec 14, 2004, at 11:16 PM, matthew ford wrote:
          > > There should be an easier (and more strightforward) way to do this.
          > > But if there is I have not found it yet.
          > > Ter can you get greedy = false to work for this case?
          >
          > The unfortunate truth is that greedy=false was an afterthought and
          > apparently not properly integrated.
          >
          > You'll note in 3.0 that this plain works:
          >
          > Q : """ (greedy=false : . )* """ ;
          >
          > I just verified the generated Java (no bytecodes in this case) and the
          > DFA. Everything is cool. It will terminate on """ and more
          > importantly, it figured out the k=6 lookahead depth out all by itself.
          > :) Hooray. Believe me when I say that I've actually gotten time to
          > think before coding this time around. 1.x was a prototype. 2.x was
          > done in a hurry to shut out JavaCC and was done while I was in a
          > start-up working like mad. I've been working on just the parsing
          > algorithm and code generator for 1.5 years now. :) I should have the
          > error handling done in a month or two. Then I clean up the tool and by
          > May we may have a parser/lexer combo that works with no support for
          > trees etc... We may also have a prototype gui.
          >
          > Ter
          > --
          > CS Professor & Grad Director, University of San Francisco
          > Creator, ANTLR Parser Generator, http://www.antlr.org
          > Cofounder, http://www.jguru.com
          > Cofounder, http://www.knowspam.net enjoy email again!
          >
          >
          >
          >
          >
          >
          > Yahoo! Groups Links
          >
          >
          >
          >
          >
          >
          >
        Your message has been successfully submitted and would be delivered to recipients shortly.