Loading ...
Sorry, an error occurred while loading the content.

3923Re: how do i skip unmatched characters?

Expand Messages
  • Ric Klaren
    Jul 2, 2001
    • 0 Attachment

      On Fri, Jun 29, 2001 at 08:47:04PM +0200, Stdiobe wrote:
      > when the lexer generated by ANTLR encounters an unmatched character,
      > it throws a TokenStreamRecognitionException which causes my lexer
      > to exit (and also my parser).

      How did you setup your exception handling in the lexer? e.g.
      defaultErrorHandler true or false? My guess is that you have it turned off?

      (As aside.. I think it is kindoff a deficiency that you can't specify an
      error handler for the nextToken rule?)

      > Does anyone know how I can skip unmatched characters in the lexer
      > by reporting the error to the user (with linenumber, etc.), and have
      > the lexer continue scanning for valid tokens.

      In my experience this is the default behaviour:


      int a�;
      line 1:6: unexpected char: 0xE9

      (the 0xE9 is devel snapshot output of 'unprintable' chars, parsing continues
      after this error)

      This is with the following relevant options:

      charVocabulary= '\u0000' .. '\u00FF';
      defaultErrorHandler = true;

      If you need defaultErrorHandler false you need to fix the error outside the
      lexer (the reason for my aside remark above). I'm not 100% sure where in
      the path to the parser the most practical place is to catch this beast.
      Relevant files would be the inputbuffer/charbuffer/sharedinputstate files.
      Maybe its better to have defaultErrorHandler set to true and override per
      rule (for as far as possible).


      ---- Ric Klaren ----- klaren@... ----- +31 53 4893722 ----
      Why don't we just invite them to dinner and massacre them all when they're
      drunk? You heard the man. There's seven hundred thousand of them.
      Ah? ... So it'd have to be something simple with pasta, then.
      --- From: Interesting Times by Terry Pratchet
    • Show all 19 messages in this topic