Loading ...
Sorry, an error occurred while loading the content.
 

exceptions in the lexer

Expand Messages
  • Paul J. Lucas
    Will the next 2.x ANTLR release have support for adding one s own exception specifications in the lexer (just like the parser)? - Paul
    Message 1 of 23 , Dec 6, 2004
      Will the next 2.x ANTLR release have support for adding one's
      own exception specifications in the lexer (just like the
      parser)?

      - Paul
    • Paul J. Lucas
      ... The former. - Paul
      Message 2 of 23 , Dec 6, 2004
        On Mon, 6 Dec 2004, Terence Parr wrote:

        > On Dec 6, 2004, at 2:12 PM, Paul J. Lucas wrote:
        >
        > >
        > > Will the next 2.x ANTLR release have support for adding one's
        > > own exception specifications in the lexer (just like the
        > > parser)?
        >
        > You mean you can't specify exceptions on lexer rules or on the
        > auto-generated nextToken rule?

        The former.

        - Paul
      • Terence Parr
        ... You mean you can t specify exceptions on lexer rules or on the auto-generated nextToken rule? Ter -- CS Professor & Grad Director, University of San
        Message 3 of 23 , Dec 6, 2004
          On Dec 6, 2004, at 2:12 PM, Paul J. Lucas wrote:

          >
          > Will the next 2.x ANTLR release have support for adding one's
          > own exception specifications in the lexer (just like the
          > parser)?

          You mean you can't specify exceptions on lexer rules or on the
          auto-generated nextToken rule?

          Ter
          --
          CS Professor & Grad Director, University of San Francisco
          Creator, ANTLR Parser Generator, http://www.antlr.org
          Cofounder, http://www.jguru.com
          Cofounder, http://www.knowspam.net enjoy email again!
        • Terence Parr
          ... Interesting. I ll think about what the heck I was thinking. Are you familiar with the filter option on the lexer grammar? Ter -- CS Professor & Grad
          Message 4 of 23 , Dec 6, 2004
            On Dec 6, 2004, at 3:38 PM, Paul J. Lucas wrote:

            >
            > On Mon, 6 Dec 2004, Terence Parr wrote:
            >
            >> On Dec 6, 2004, at 2:12 PM, Paul J. Lucas wrote:
            >>
            >>>
            >>> Will the next 2.x ANTLR release have support for adding one's
            >>> own exception specifications in the lexer (just like the
            >>> parser)?
            >>
            >> You mean you can't specify exceptions on lexer rules or on the
            >> auto-generated nextToken rule?
            >
            > The former.

            Interesting. I'll think about what the heck I was thinking. Are you
            familiar with the filter option on the lexer grammar?

            Ter
            --
            CS Professor & Grad Director, University of San Francisco
            Creator, ANTLR Parser Generator, http://www.antlr.org
            Cofounder, http://www.jguru.com
            Cofounder, http://www.knowspam.net enjoy email again!
          • Paul J. Lucas
            ... Yes in the sense that I ve read that bit of the documentation, but I ve never used it. I don t want my entire lexer to be a filter since I need it to be a
            Message 5 of 23 , Dec 6, 2004
              On Mon, 6 Dec 2004, Terence Parr wrote:

              > Are you familiar with the filter option on the lexer grammar?

              Yes in the sense that I've read that bit of the documentation,
              but I've never used it.

              I don't want my entire lexer to be a filter since I need it to
              be a real lexer for a parser.

              My particular case at hand is dealing with pragmas that can
              appear anywhere comments can. Since comments are most easily
              handled in the lexer, I handle pragmas there to for the same
              reason. The problem is what to do if there's an error in a
              pragma. The lexer can't throw user-defined exceptions. (I
              might be able to work around it by hiding the real exception
              inside a Runtime exception, but that's ugly.

              - Paul
            • Terence Parr
              ... Ah. An example makes everything clear. What precisely do you want to happen when there is an error? I guess at minimum an error, but the question is:
              Message 6 of 23 , Dec 6, 2004
                On Dec 6, 2004, at 4:10 PM, Paul J. Lucas wrote:

                >
                > On Mon, 6 Dec 2004, Terence Parr wrote:
                >
                >> Are you familiar with the filter option on the lexer grammar?
                >
                > Yes in the sense that I've read that bit of the documentation,
                > but I've never used it.
                >
                > I don't want my entire lexer to be a filter since I need it to
                > be a real lexer for a parser.
                >
                > My particular case at hand is dealing with pragmas that can
                > appear anywhere comments can. Since comments are most easily
                > handled in the lexer, I handle pragmas there to for the same
                > reason. The problem is what to do if there's an error in a
                > pragma. The lexer can't throw user-defined exceptions. (I
                > might be able to work around it by hiding the real exception
                > inside a Runtime exception, but that's ugly.

                Ah. An example makes everything clear. What precisely do you want to
                happen when there is an error? I guess at minimum an error, but the
                question is: how to trap it? Recovery is also an issue. Let me look
                at the output for a second...

                Ok, generated nextToken looks like it catches and rethrows:

                catch (RecognitionException e) {
                throw new TokenStreamRecognitionException(e);
                }

                Shoot. Well, the token stream exception can be caught easily in a
                TokenStream wrapped around the Lexer. You can even ask it to retry for
                another token.

                Ok, for a REAL hack, you could wrap a reference to a lexer rule with
                actions containing a "try {" and then "} catch...", though ANTLR might
                get pissed about the unbalanced curlies. Might have to escape them
                with \.

                Hmm...I'd probably go for the TokenStream filter that catches the
                exception. Easiest thing.

                Ter
                --
                CS Professor & Grad Director, University of San Francisco
                Creator, ANTLR Parser Generator, http://www.antlr.org
                Cofounder, http://www.jguru.com
                Cofounder, http://www.knowspam.net enjoy email again!
              • Don Caton
                ... Ter: What s the point of this? TokenStreamRecognitionException simply wraps a RecognitionException and does not appear to add any useful functionality. In
                Message 7 of 23 , Dec 6, 2004
                  > -----Original Message-----
                  > From: Terence Parr [mailto:parrt@...]
                  > Sent: Monday, December 06, 2004 8:03 PM
                  > To: antlr-interest@yahoogroups.com
                  > Subject: Re: [antlr-interest] exceptions in the lexer
                  >
                  > Ok, generated nextToken looks like it catches and rethrows:
                  >
                  > catch (RecognitionException e) {
                  > throw new TokenStreamRecognitionException(e);
                  > }
                  >

                  Ter:

                  What's the point of this? TokenStreamRecognitionException simply wraps a
                  RecognitionException and does not appear to add any useful functionality.
                  In fact, it removes functionality by preventing access to the original
                  RecognitionException because it is stored in a private member of
                  TokenStreamRecognitionException.

                  This makes it impossible to retrieve the line and column information from
                  the RecognitionException. Which in turn makes it impossible to display
                  anything other than the hard coded error message using the toString() method
                  provided by TokenStreamRecognitionException.

                  TokenStreamRecognitionException itself is a do-nothing subclass of
                  TokenStreamException, which in turn is a do-nothing subclass of
                  AntlrException. Why the complexity? Are there some backwards-compatibility
                  reasons for this?

                  At the risk of breaking existing code, I'd like to see a simplified error
                  reporting mechanism in 2.7.5 where a single exception type is thrown,
                  perhaps with a "reason" code or enumeration. Then we only need to handle a
                  single exception type. Or at least, have every exception inherit from a
                  base class, don't wrap one exception up in another.

                  Just my 2 cents.

                  Don
                • matthew ford
                  Hey, Just extend you user-defined exceptions from RuntimeException and throw them. You catch all your RuntimeExceptions don t you? Remember
                  Message 8 of 23 , Dec 7, 2004
                    Hey, Just extend you user-defined exceptions from RuntimeException and throw
                    them.
                    You catch all your RuntimeExceptions don't you?
                    Remember OutOfMemoryException is not necessarly fatal.
                    matthew
                    ----- Original Message -----
                    From: "Paul J. Lucas" <pauljlucas@...>
                    To: <antlr-interest@yahoogroups.com>
                    Sent: Tuesday, December 07, 2004 11:10 AM
                    Subject: Re: [antlr-interest] exceptions in the lexer


                    >
                    > On Mon, 6 Dec 2004, Terence Parr wrote:
                    >
                    > > Are you familiar with the filter option on the lexer grammar?
                    >
                    > Yes in the sense that I've read that bit of the documentation,
                    > but I've never used it.
                    >
                    > I don't want my entire lexer to be a filter since I need it to
                    > be a real lexer for a parser.
                    >
                    > My particular case at hand is dealing with pragmas that can
                    > appear anywhere comments can. Since comments are most easily
                    > handled in the lexer, I handle pragmas there to for the same
                    > reason. The problem is what to do if there's an error in a
                    > pragma. The lexer can't throw user-defined exceptions. (I
                    > might be able to work around it by hiding the real exception
                    > inside a Runtime exception, but that's ugly.
                    >
                    > - Paul
                    >
                    >
                    >
                    >
                    > Yahoo! Groups Links
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                  • Harald Mueller
                    ... He explicitly didn t want to do this - but I also do it (using a WrappingRuntimeException which I need at other places anyway (e.g. for the Runnable.run()
                    Message 9 of 23 , Dec 7, 2004
                      >
                      > Hey, Just extend you user-defined exceptions from RuntimeException and
                      > throw
                      > them.

                      He explicitly didn't want to do this - but I also do it (using a
                      WrappingRuntimeException which I need at other places anyway (e.g. for the
                      Runnable.run() method)).

                      > You catch all your RuntimeExceptions don't you?
                      > Remember OutOfMemoryException is not necessarly fatal.

                      This is not an Exception, but an Error - you would never catch this.

                      Regards
                      Harald M. Müller

                      > matthew
                      > ----- Original Message -----
                      > From: "Paul J. Lucas" <pauljlucas@...>
                      > To: <antlr-interest@yahoogroups.com>
                      > Sent: Tuesday, December 07, 2004 11:10 AM
                      > Subject: Re: [antlr-interest] exceptions in the lexer
                      >
                      >
                      > >
                      > > On Mon, 6 Dec 2004, Terence Parr wrote:
                      > >
                      > > > Are you familiar with the filter option on the lexer grammar?
                      > >
                      > > Yes in the sense that I've read that bit of the documentation,
                      > > but I've never used it.
                      > >
                      > > I don't want my entire lexer to be a filter since I need it to
                      > > be a real lexer for a parser.
                      > >
                      > > My particular case at hand is dealing with pragmas that can
                      > > appear anywhere comments can. Since comments are most easily
                      > > handled in the lexer, I handle pragmas there to for the same
                      > > reason. The problem is what to do if there's an error in a
                      > > pragma. The lexer can't throw user-defined exceptions. (I
                      > > might be able to work around it by hiding the real exception
                      > > inside a Runtime exception, but that's ugly.
                      > >
                      > > - Paul
                      > >
                      > >
                      > >
                      > >
                      > > Yahoo! Groups Links
                      > >
                      > >
                      > >
                      > >
                      > >
                      > >
                      > >
                      >
                      >
                      >
                      >
                      > Yahoo! Groups Links
                      >
                      >
                      >
                      >
                      >
                      >
                      >

                      --
                      NEU +++ DSL Komplett von GMX +++ http://www.gmx.net/de/go/dsl
                      GMX DSL-Netzanschluss + Tarif zum supergünstigen Komplett-Preis!
                    • matthew ford
                      Well JEdit catches this and recovers. Why can t I? matthew ... From: Harald Mueller To: Sent:
                      Message 10 of 23 , Dec 7, 2004
                        Well JEdit catches this and recovers.
                        Why can't I?
                        matthew

                        ----- Original Message -----
                        From: "Harald Mueller" <harald.m.mueller@...>
                        To: <antlr-interest@yahoogroups.com>
                        Sent: Tuesday, December 07, 2004 8:48 PM
                        Subject: Re: [antlr-interest] exceptions in the lexer


                        >
                        > >
                        > > Hey, Just extend you user-defined exceptions from RuntimeException and
                        > > throw
                        > > them.
                        >
                        > He explicitly didn't want to do this - but I also do it (using a
                        > WrappingRuntimeException which I need at other places anyway (e.g. for the
                        > Runnable.run() method)).
                        >
                        > > You catch all your RuntimeExceptions don't you?
                        > > Remember OutOfMemoryException is not necessarly fatal.
                        >
                        > This is not an Exception, but an Error - you would never catch this.
                        >
                        > Regards
                        > Harald M. Müller
                        >
                        > > matthew
                        > > ----- Original Message -----
                        > > From: "Paul J. Lucas" <pauljlucas@...>
                        > > To: <antlr-interest@yahoogroups.com>
                        > > Sent: Tuesday, December 07, 2004 11:10 AM
                        > > Subject: Re: [antlr-interest] exceptions in the lexer
                        > >
                        > >
                        > > >
                        > > > On Mon, 6 Dec 2004, Terence Parr wrote:
                        > > >
                        > > > > Are you familiar with the filter option on the lexer grammar?
                        > > >
                        > > > Yes in the sense that I've read that bit of the documentation,
                        > > > but I've never used it.
                        > > >
                        > > > I don't want my entire lexer to be a filter since I need it to
                        > > > be a real lexer for a parser.
                        > > >
                        > > > My particular case at hand is dealing with pragmas that can
                        > > > appear anywhere comments can. Since comments are most easily
                        > > > handled in the lexer, I handle pragmas there to for the same
                        > > > reason. The problem is what to do if there's an error in a
                        > > > pragma. The lexer can't throw user-defined exceptions. (I
                        > > > might be able to work around it by hiding the real exception
                        > > > inside a Runtime exception, but that's ugly.
                        > > >
                        > > > - Paul
                        > > >
                        > > >
                        > > >
                        > > >
                        > > > Yahoo! Groups Links
                        > > >
                        > > >
                        > > >
                        > > >
                        > > >
                        > > >
                        > > >
                        > >
                        > >
                        > >
                        > >
                        > > Yahoo! Groups Links
                        > >
                        > >
                        > >
                        > >
                        > >
                        > >
                        > >
                        >
                        > --
                        > NEU +++ DSL Komplett von GMX +++ http://www.gmx.net/de/go/dsl
                        > GMX DSL-Netzanschluss + Tarif zum supergünstigen Komplett-Preis!
                        >
                        >
                        >
                        > Yahoo! Groups Links
                        >
                        >
                        >
                        >
                        >
                        >
                        >
                      • Paul J. Lucas
                        ... I m aware of that, but prefer not to do it. - Paul
                        Message 11 of 23 , Dec 7, 2004
                          On Tue, 7 Dec 2004, matthew ford wrote:

                          > Just extend you user-defined exceptions from RuntimeException and throw
                          > them.

                          I'm aware of that, but prefer not to do it.

                          - Paul
                        • Terence Parr
                          ... I believe I wanted TokenStream.nextToken() to be independent of the normal recognition errors etc... found in antlr s lexers; might be using another tool.
                          Message 12 of 23 , Dec 7, 2004
                            On Dec 6, 2004, at 5:58 PM, Don Caton wrote:
                            >> Ok, generated nextToken looks like it catches and rethrows:
                            >>
                            >> catch (RecognitionException e) {
                            >> throw new TokenStreamRecognitionException(e);
                            >> }
                            >>
                            >
                            > Ter:
                            >
                            > What's the point of this? TokenStreamRecognitionException simply
                            > wraps a
                            > RecognitionException and does not appear to add any useful
                            > functionality.

                            I believe I wanted TokenStream.nextToken() to be independent of the
                            normal recognition errors etc... found in antlr's lexers; might be
                            using another tool.

                            public interface TokenStream {
                            public Token nextToken() throws TokenStreamException;
                            }

                            > In fact, it removes functionality by preventing access to the original
                            > RecognitionException because it is stored in a private member of
                            > TokenStreamRecognitionException.

                            Ack! Heh, wait, looks public:

                            public class TokenStreamRecognitionException extends
                            TokenStreamException {
                            public RecognitionException recog;
                            ....
                            }

                            > This makes it impossible to retrieve the line and column information
                            > from
                            > the RecognitionException. Which in turn makes it impossible to display
                            > anything other than the hard coded error message using the toString()
                            > method
                            > provided by TokenStreamRecognitionException.
                            >
                            > TokenStreamRecognitionException itself is a do-nothing subclass of
                            > TokenStreamException, which in turn is a do-nothing subclass of
                            > AntlrException. Why the complexity? Are there some
                            > backwards-compatibility
                            > reasons for this?

                            Do nothing, but mean something ;)

                            > At the risk of breaking existing code, I'd like to see a simplified
                            > error
                            > reporting mechanism in 2.7.5 where a single exception type is thrown,
                            > perhaps with a "reason" code or enumeration. Then we only need to
                            > handle a
                            > single exception type. Or at least, have every exception inherit from
                            > a
                            > base class, don't wrap one exception up in another.

                            Good thoughts, Don. I'm afraid of breaking stuff for 2.7.5 though.
                            This will be my next topic (error recovery) for 3.0 as I've got 3.0
                            good enough to handle Java parsing/lexing now.

                            Ter
                            --
                            CS Professor & Grad Director, University of San Francisco
                            Creator, ANTLR Parser Generator, http://www.antlr.org
                            Cofounder, http://www.jguru.com
                            Cofounder, http://www.knowspam.net enjoy email again!
                          • Don Caton
                            ... I don t do Java ;). In the C++ version, it s private. Perhaps that is just a porting error then. Don
                            Message 13 of 23 , Dec 7, 2004
                              > -----Original Message-----
                              > From: Terence Parr [mailto:parrt@...]
                              > Sent: Tuesday, December 07, 2004 2:13 PM
                              > To: antlr-interest@yahoogroups.com
                              > Subject: Re: [antlr-interest] exceptions in the lexer
                              >
                              >
                              > Ack! Heh, wait, looks public:
                              >
                              > public class TokenStreamRecognitionException extends
                              > TokenStreamException {
                              > public RecognitionException recog;
                              > ....
                              > }

                              I don't do Java ;). In the C++ version, it's private. Perhaps that is just
                              a porting error then.

                              Don
                            • Terence Parr
                              ... Probably. Yeah, I ve yet to find a case where private is actually what I want. I m a protected man myself ;) Ter -- CS Professor & Grad Director,
                              Message 14 of 23 , Dec 7, 2004
                                On Dec 7, 2004, at 11:16 AM, Don Caton wrote:

                                >
                                >> -----Original Message-----
                                >> From: Terence Parr [mailto:parrt@...]
                                >> Sent: Tuesday, December 07, 2004 2:13 PM
                                >> To: antlr-interest@yahoogroups.com
                                >> Subject: Re: [antlr-interest] exceptions in the lexer
                                >>
                                >>
                                >> Ack! Heh, wait, looks public:
                                >>
                                >> public class TokenStreamRecognitionException extends
                                >> TokenStreamException {
                                >> public RecognitionException recog;
                                >> ....
                                >> }
                                >
                                > I don't do Java ;). In the C++ version, it's private. Perhaps that
                                > is just
                                > a porting error then.

                                Probably. Yeah, I've yet to find a case where private is actually what
                                I want. I'm a protected man myself ;)

                                Ter
                                --
                                CS Professor & Grad Director, University of San Francisco
                                Creator, ANTLR Parser Generator, http://www.antlr.org
                                Cofounder, http://www.jguru.com
                                Cofounder, http://www.knowspam.net enjoy email again!
                              • Ric Klaren
                                On Tue, 7 Dec 2004 14:16:15 -0500, Don Caton ... The C++ version has some proxy methods to the recognition exception. A bit nicer than a public accessible
                                Message 15 of 23 , Dec 7, 2004
                                  On Tue, 7 Dec 2004 14:16:15 -0500, Don Caton
                                  <dcaton@...> wrote:
                                  > > Ack! Heh, wait, looks public:
                                  > >
                                  > > public class TokenStreamRecognitionException extends
                                  > > TokenStreamException {
                                  > > public RecognitionException recog;
                                  > > ....
                                  > > }
                                  >
                                  > I don't do Java ;). In the C++ version, it's private. Perhaps that is just
                                  > a porting error then.

                                  The C++ version has some proxy methods to the recognition exception. A
                                  bit nicer than a public accessible attribute. (Not sure if that's
                                  already in 2.7.4)

                                  Cheers,

                                  Ric
                                • Paul J. Lucas
                                  ... I ll add to the complaint about TokenStreamRecognitionException. The default parser recovery code only catches RecognitionException, so if a
                                  Message 16 of 23 , Dec 10, 2004
                                    --- In antlr-interest@yahoogroups.com, Terence Parr <parrt@c...> wrote:
                                    >
                                    > On Dec 6, 2004, at 5:58 PM, Don Caton wrote:
                                    > > What's the point of this? TokenStreamRecognitionException simply
                                    > > wraps a
                                    > > RecognitionException and does not appear to add any useful
                                    > > functionality.
                                    >
                                    > I believe I wanted TokenStream.nextToken() to be independent of the
                                    > normal recognition errors etc... found in antlr's lexers; might be
                                    > using another tool.

                                    I'll add to the complaint about TokenStreamRecognitionException. The default parser
                                    recovery code only catches RecognitionException, so if a
                                    TokenStreamRecognitionException is thrown, it bypasses my nice recovery mechanism.
                                    This apparently means that, every place I catch a RecognitionException to recover, I must
                                    also now catch a TokenStreamRecognitionException.

                                    Sigh...

                                    Either TokenStreamRecognitionException needs to go away or the auto-generated parser
                                    recovery code needs to catch TokenStreamRecognitionException in addition to
                                    RecognitionException (and do the same recovery stuff).

                                    - Paul
                                  • Terence Parr
                                    ... Hi John..er..Paul. ;) Weird. It does look like TokenStreamRecognitionException is outside of the RecognitionException. Hmm...I know I did that for a
                                    Message 17 of 23 , Dec 10, 2004
                                      On Dec 10, 2004, at 1:18 PM, Paul J. Lucas wrote:

                                      >
                                      >
                                      > --- In antlr-interest@yahoogroups.com, Terence Parr <parrt@c...> wrote:
                                      >>
                                      >> On Dec 6, 2004, at 5:58 PM, Don Caton wrote:
                                      >>> What's the point of this? TokenStreamRecognitionException simply
                                      >>> wraps a
                                      >>> RecognitionException and does not appear to add any useful
                                      >>> functionality.
                                      >>
                                      >> I believe I wanted TokenStream.nextToken() to be independent of the
                                      >> normal recognition errors etc... found in antlr's lexers; might be
                                      >> using another tool.
                                      >
                                      > I'll add to the complaint about TokenStreamRecognitionException. The
                                      > default parser
                                      > recovery code only catches RecognitionException, so if a
                                      > TokenStreamRecognitionException is thrown, it bypasses my nice
                                      > recovery mechanism.
                                      > This apparently means that, every place I catch a RecognitionException
                                      > to recover, I must
                                      > also now catch a TokenStreamRecognitionException.
                                      >
                                      > Sigh...

                                      Hi John..er..Paul. ;)

                                      Weird. It does look like TokenStreamRecognitionException is outside of
                                      the RecognitionException. Hmm...I know I did that for a reason, but it
                                      escapes me. Regardless, your argument holds that it's a pain!

                                      > Either TokenStreamRecognitionException needs to go away or the
                                      > auto-generated parser
                                      > recovery code needs to catch TokenStreamRecognitionException in
                                      > addition to
                                      > RecognitionException (and do the same recovery stuff).

                                      Hm...perhaps I should just make it catch ANTLRException, the root,
                                      instead?

                                      Ter

                                      >
                                      > - Paul
                                      >
                                      >
                                      >
                                      >
                                      >
                                      >
                                      > Yahoo! Groups Links
                                      >
                                      >
                                      >
                                      >
                                      >
                                      >
                                      >
                                      >
                                      --
                                      CS Professor & Grad Director, University of San Francisco
                                      Creator, ANTLR Parser Generator, http://www.antlr.org
                                      Cofounder, http://www.jguru.com
                                      Cofounder, http://www.knowspam.net enjoy email again!
                                    • Paul J. Lucas
                                      ... That would work also (and should fix the problem once and for all). - Paul
                                      Message 18 of 23 , Dec 10, 2004
                                        On Fri, 10 Dec 2004, Terence Parr wrote:

                                        > Hm...perhaps I should just make it catch ANTLRException, the root,
                                        > instead?

                                        That would work also (and should fix the problem once and for
                                        all).

                                        - Paul
                                      • Terence Parr
                                        ... Agreed, though I m afraid it would break something. I ll add to list to examine for 2.7.5, which I hope to jump on next week around thursday. Ter -- CS
                                        Message 19 of 23 , Dec 10, 2004
                                          On Dec 10, 2004, at 2:25 PM, Paul J. Lucas wrote:

                                          >
                                          > On Fri, 10 Dec 2004, Terence Parr wrote:
                                          >
                                          >> Hm...perhaps I should just make it catch ANTLRException, the root,
                                          >> instead?
                                          >
                                          > That would work also (and should fix the problem once and for
                                          > all).

                                          Agreed, though I'm afraid it would break something. I'll add to list
                                          to examine for 2.7.5, which I hope to jump on next week around
                                          thursday.

                                          Ter
                                          --
                                          CS Professor & Grad Director, University of San Francisco
                                          Creator, ANTLR Parser Generator, http://www.antlr.org
                                          Cofounder, http://www.jguru.com
                                          Cofounder, http://www.knowspam.net enjoy email again!
                                        • Terence Parr
                                          ... Oh, I think I remember now. Definitely had to do with TokenRetryException. I wanted to be able to blast out of the lexer entirely on an exception to
                                          Message 20 of 23 , Dec 10, 2004
                                            On Dec 10, 2004, at 2:25 PM, Paul J. Lucas wrote:
                                            > On Fri, 10 Dec 2004, Terence Parr wrote:
                                            >
                                            >> Hm...perhaps I should just make it catch ANTLRException, the root,
                                            >> instead?
                                            >
                                            > That would work also (and should fix the problem once and for
                                            > all).

                                            Oh, I think I remember now. Definitely had to do with
                                            TokenRetryException. I wanted to be able to blast out of the lexer
                                            entirely on an exception to force the a token stream switch to flip to
                                            another lexer and retry. If I made that part of the same hierarchy
                                            under RecognitionException then you couldn't do the retry. I
                                            specifically made it separate. Hmm...seems like we need it.

                                            From the comment on the interface, "Anything that goes wrong while
                                            generating a stream of tokens", I think I meant to be different than
                                            recognition issues. For the life of me though, I can't remember
                                            expecting lexer *recognition* exceptions to make it to the parser.
                                            Yet, that is what the code says. Bizarre. The lexer should just look
                                            for another token.

                                            Ok, i'll look into it.
                                            Ter
                                            --
                                            CS Professor & Grad Director, University of San Francisco
                                            Creator, ANTLR Parser Generator, http://www.antlr.org
                                            Cofounder, http://www.jguru.com
                                            Cofounder, http://www.knowspam.net enjoy email again!
                                          • Colm McHugh
                                            ... Might that something be related to syntactic predicate? I ve been grappling with this exact same problem for the past week, namely trapping bad character
                                            Message 21 of 23 , Dec 10, 2004
                                              --- Terence Parr <parrt@...> wrote:
                                              >
                                              >
                                              > On Dec 10, 2004, at 2:25 PM, Paul J. Lucas wrote:
                                              >
                                              > >
                                              > > On Fri, 10 Dec 2004, Terence Parr wrote:
                                              > >
                                              > >> Hm...perhaps I should just make it catch
                                              > ANTLRException, the root,
                                              > >> instead?
                                              > >
                                              > > That would work also (and should fix the problem
                                              > once and for
                                              > > all).
                                              >
                                              > Agreed, though I'm afraid it would break something.
                                              > I'll add to list
                                              > to examine for 2.7.5, which I hope to jump on next
                                              > week around
                                              > thursday.
                                              >
                                              Might that 'something' be related to syntactic
                                              predicate?

                                              I've been grappling with this exact same problem for
                                              the past week, namely trapping bad character (or
                                              lexical) errors.

                                              I switched all catches of TokenRecognitionException in
                                              my parser to catch ANTLRException (I want to do my own
                                              exception handling), and that traps all bad characters
                                              except for one case, and that is when a bad character
                                              is found while in syntactic predicate guess mode; the
                                              exception does not get trapped and causes the process
                                              to abort [this is bad]. The same bad character error
                                              is trapped when it's not part of a syntactic
                                              predicate, hence my finger-pointing at syn preds.

                                              I've looked at the generated code for the parser, and
                                              a catch block is only executed if guessing is not on,
                                              otherwise it does a 'throw'. Why is this ? Is there a
                                              good reason to not do the user's exception-handling if
                                              we are in guess mode ?

                                              My brain is refusing to delve further into the problem
                                              (it _is_ 4:30pm on Friday) but let me know if there is
                                              anything that vaguely makes sense in this line of
                                              thought.

                                              Cheers!
                                              Colm.





                                              __________________________________
                                              Do you Yahoo!?
                                              Take Yahoo! Mail with you! Get it on your mobile phone.
                                              http://mobile.yahoo.com/maildemo
                                            • Terence Parr
                                              ... Hi Colm. :) Nice to hear from you. Nope...not syn pred stuff. Recognition problems should be caught by the syn pred and ignored; then the next viable
                                              Message 22 of 23 , Dec 11, 2004
                                                On Dec 10, 2004, at 5:11 PM, Colm McHugh wrote:
                                                >> Agreed, though I'm afraid it would break something.
                                                >> I'll add to list
                                                >> to examine for 2.7.5, which I hope to jump on next
                                                >> week around
                                                >> thursday.
                                                >>
                                                > Might that 'something' be related to syntactic
                                                > predicate?
                                                Hi Colm. :) Nice to hear from you.

                                                Nope...not syn pred stuff. Recognition problems should be caught by
                                                the syn pred and ignored; then the next viable alternative should be
                                                attempted.

                                                >
                                                > I've been grappling with this exact same problem for
                                                > the past week, namely trapping bad character (or
                                                > lexical) errors.
                                                >
                                                > I switched all catches of TokenRecognitionException in
                                                > my parser to catch ANTLRException (I want to do my own
                                                > exception handling), and that traps all bad characters
                                                > except for one case, and that is when a bad character
                                                > is found while in syntactic predicate guess mode; the
                                                > exception does not get trapped and causes the process
                                                > to abort [this is bad]. The same bad character error
                                                > is trapped when it's not part of a syntactic
                                                > predicate, hence my finger-pointing at syn preds.
                                                >
                                                Ah. Well, no the syn preds are using recognition errors to say "oh,
                                                that alt won't match; try again with next alt."

                                                > I've looked at the generated code for the parser, and
                                                > a catch block is only executed if guessing is not on,
                                                > otherwise it does a 'throw'. Why is this ? Is there a
                                                > good reason to not do the user's exception-handling if
                                                > we are in guess mode ?
                                                >
                                                Yes, all catches are turned off except for one generated for the syn
                                                pred.
                                                >
                                                Ter
                                                --
                                                CS Professor & Grad Director, University of San Francisco
                                                Creator, ANTLR Parser Generator, http://www.antlr.org
                                                Cofounder, http://www.jguru.com
                                                Cofounder, http://www.knowspam.net enjoy email again!
                                              • Colm McHugh
                                                ... Hey Ter, thanks for the response; I haven t completely taken in your comments, but the consequence of an exception happenning in syn pred guess mode is
                                                Message 23 of 23 , Dec 13, 2004
                                                  > Hi Colm. :) Nice to hear from you.
                                                  >
                                                  > Nope...not syn pred stuff. Recognition problems
                                                  > should be caught by
                                                  > the syn pred and ignored; then the next viable
                                                  > alternative should be
                                                  > attempted.
                                                  >

                                                  Hey Ter, thanks for the response; I haven't completely
                                                  taken in your comments, but the consequence of an
                                                  exception happenning in syn pred guess mode is that I
                                                  need to put a 'catch (...)' around the code that
                                                  creates and calls my parser object. This is really
                                                  just to catch the 'throw;' that the parser does when
                                                  it's in guess mode. [I'm using C++ BTW]

                                                  I guess I'm just sharing a mistaken assumption I had
                                                  been making, which is that it's possible to catch all
                                                  exceptions within the parser (.g).

                                                  Good work on the 3.0 lexer by the way! Hopefully folks
                                                  won't be so concerned about using a non-ANTLR lexer
                                                  with 3.0. Just out of interest, I've been working on a
                                                  project where I've replaced an ancient hand-written
                                                  parser with ANTLR 2.7.4 parser and lexer and the
                                                  performance is comparable (my ANTLR written
                                                  replacement stuff is about 40 secs slower in a 15
                                                  minute test, and I haven't done any profiling or
                                                  optimization yet).

                                                  Cheers,
                                                  Colm.



                                                  __________________________________
                                                  Do you Yahoo!?
                                                  Dress up your holiday email, Hollywood style. Learn more.
                                                  http://celebrity.mail.yahoo.com
                                                Your message has been successfully submitted and would be delivered to recipients shortly.