Loading ...
Sorry, an error occurred while loading the content.

RE: AST Tree construction.

Expand Messages
  • Ken Lidster
    Sorry. I haven t worked with ANTLR for about 8 months. That syntax should be { ## = #([SECT], #([PARA], p), #([DOCSEC], d), #([HDRFTR], h)); } Also, if you
    Message 1 of 18 , Dec 7, 1999
    • 0 Attachment
      Sorry. I haven't worked with ANTLR for about 8 months. That syntax should be

      { ## = #([SECT], #([PARA], p), #([DOCSEC], d), #([HDRFTR], h)); }

      Also, if you want to print out the tree at some point you should probably
      put in identifying text for the token types (i.e., [SECT,"SECT"],
      [PARA,"PARA"], etc.).

      Ken

      -----Original Message-----
      From: kevin.burton@... [mailto:kevin.burton@...]
      Sent: Tuesday, December 07, 1999 11:51 AM
      To: antlr-interest@onelist.com
      Subject: RE: [antlr-interest] AST Tree construction.


      From: kevin.burton@...

      Ken,

      Thanks for your comments.

      Is this the correct syntax? I tried it and am getting "Syntax error in
      action: Error: line(62), mismatched char: '#' expected ')'

      Kevin

      > -----Original Message-----
      > From: Ken Lidster [SMTP:Ken.Lidster@...]
      > Sent: Tuesday, December 07, 1999 1:18 PM
      > To: 'antlr-interest@onelist.com'
      > Subject: RE: [antlr-interest] AST Tree construction.
      >
      > From: Ken Lidster <Ken.Lidster@...>
      >
      > Kevin,
      >
      > I'm certainly not an expert, but you might want to consider something like
      >
      > sectionList
      > : section (SECT section)*
      > ;
      > section!
      > : (d:docsecfmtitems)? (h:hdrftritems)? p:paraitems
      > { ## = #(SECT #(PARA #p) #(DOCSEC #d) #(HDRFTR #h)); }
      > ;
      >
      > This would give you a fifo list of one or more SECT nodes, where each SECT
      > node has a PARA, DOCSEC and HDRFTR node entry.
      >
      > By the way, you can get confusing results when you build AST nodes with
      > entries that can be either empty or a list. I've found that its usually
      > easier to build a separate AST node for these, as with the PARA, DOCSEC
      > and
      > HDRFTR nodes above.
      >
      > Ken
      > ken@...
      >
      > -----Original Message-----
      > From: kevin.burton@... [mailto:kevin.burton@...]
      > Sent: Tuesday, December 07, 1999 10:04 AM
      > To: antlr-interest@onelist.com
      > Subject: RE: [antlr-interest] AST Tree construction.
      >
      >
      > From: kevin.burton@...
      >
      > I really do not want #(#(df), #(hf), #(p)). I agree that it does not make
      > sense to have a tree as a root node. Is what I want is to form three
      > separate trees, one for df, one for hf, and one for p. For one I don't
      > know
      > how to form a tree given a list of input, and I don't know how to
      > correctly
      > handle the recursion that is implicit with the 'section' rule below. If I
      > need to create a special node as the name of the section that would be
      > nice
      > to, but I am not sure how to do that either. Maybe the grammar is unclear.
      > Maybe it should be something like:
      >
      > sectionList
      > : section (SECT sectionList)?
      > ;
      >
      > section
      > : (docsecfmtitems)? (hdrftritems)? paraitems
      > ;
      >
      > Would this be easier to generate a tree from? But, again for the section
      > rule I would probably need to create a node that could act as the root.
      >
      > Thank you for you continued help.
      >
      > Kevin Burton
      > Kevin.Burton@...
      >
      > > -----Original Message-----
      > > From: parrt@... [SMTP:parrt@...]
      > > Sent: Tuesday, December 07, 1999 11:52 AM
      > > To: antlr-interest@onelist.com
      > > Subject: RE: [antlr-interest] AST Tree construction.
      > >
      > > From: <parrt@...>
      > >
      > > kevin.burton@... writes:
      > > > section!
      > > > : ((df:docsecfmtitems)?
      > > >(hf:hdrftritems)? p:paraitems) (SECT^ section)?
      > > > { #section = #(#(df), #(hf), #(p));
      > > >}
      > >
      > > This looks weird to me. Get rid of the extra parens. #(...) is a
      > > tree. #label is a reference to a tree. Also, you use ^, but you have
      > > ! on the rule saying not to autogen the tree...not sure what would
      > > happen. Use
      > >
      > > #section = #(#df, #hf, #p);
      > >
      > > Actually #df as root won't work probably as it's a tree not a single
      > > node. That makes no sense to have a tree as a root.
      > >
      > >
      > > Hope this helps.
      > >
      > > Ter
      > >
      > >
      >
      >
    • Ken Lidster
      Kevin, No problem. I do recommend, though, that you change sectionList ... ; to sectionList ... ; as it eliminates the recursion and does simple AST list
      Message 2 of 18 , Dec 7, 1999
      • 0 Attachment
        Kevin,

        No problem. I do recommend, though, that you change

        sectionList
        : section (SECT sectionList)?
        ;
        to
        sectionList
        : section (SECT section)*
        ;

        as it eliminates the recursion and does simple AST list construction.

        As to context-sensitive tokens, there are a couple of options. The simplest,
        of course, is to redefine your grammar so that the nondeterminism is
        removed. For example, you said that there is a set of tokens that are used
        by both the 'paraitems' and 'docsetfmtitems' rules. Can you move these into
        a common rule that is referenced by both?

        The other option is the new 'Lexer States' mechanism that is available with
        2.6.0. I haven't used it myself, but the documentation (see "streams.html"
        in the doc directory) indicates this could be a promising solution to this
        class of problem.

        Ken
        ken@...

        -----Original Message-----
        From: kevin.burton@... [mailto:kevin.burton@...]
        Sent: Tuesday, December 07, 1999 1:35 PM
        To: antlr-interest@onelist.com
        Subject: RE: [antlr-interest] AST Tree construction.


        From: kevin.burton@...

        Ken,

        Thanks to your comments I was able to put together the following code that
        does mostly what I want.

        tokens {
        SECTION;
        DOCSEC;
        HDRFTR;
        PARA;
        }

        . . . . .

        sectionList
        : section (SECT sectionList)?
        ;

        section!
        : d:docsecfmtitems h:hdrftritems p:paraitems
        { #section = #([SECTION, "section"],
        ([DOCSEC, "docsec"], #d), ([HDRFTR, "hdrftr"], #h), ([PARA, "para"], #p) );
        }
        | d:docsecfmtitems p:paraitems
        { #section = #([SECTION, "section"],
        ([DOCSEC, "docsec"], #d), ([PARA, "para"], #p) ); }
        | h:hdrftritems p:paraitems
        { #section = #([SECTION, "section"],
        ([HDRFTR, "hdrftr"], #h), ([PARA, "para"], #p) ); }
        | p:paraitems
        { #section = #([SECTION, "section"], ([PARA,
        "para"], #p) ); }
        ;

        I get a "section" tree that contains at most three trees. This is a great
        step forward. Thank you.

        The problem is that doing this introduced a nondeterminism warning into my
        grammar. It turns out that 'paraitems' can contain tokens that are defined
        in 'docsecfmtitems'. Is there a mechanism that I can distinquish between
        tokens based on context? Any suggestions as to how I can resolve these
        nondeterminism warnings (errors)?

        Thanks again.

        Kevin

        > -----Original Message-----
        > From: kevin.burton@... [SMTP:kevin.burton@...]
        > Sent: Tuesday, December 07, 1999 1:51 PM
        > To: antlr-interest@onelist.com
        > Subject: RE: [antlr-interest] AST Tree construction.
        >
        > From: kevin.burton@...
        >
        > Ken,
        >
        > Thanks for your comments.
        >
        > Is this the correct syntax? I tried it and am getting "Syntax error in
        > action: Error: line(62), mismatched char: '#' expected ')'
        >
        > Kevin
        >
        > > -----Original Message-----
        > > From: Ken Lidster [SMTP:Ken.Lidster@...]
        > > Sent: Tuesday, December 07, 1999 1:18 PM
        > > To: 'antlr-interest@onelist.com'
        > > Subject: RE: [antlr-interest] AST Tree construction.
        > >
        > > From: Ken Lidster <Ken.Lidster@...>
        > >
        > > Kevin,
        > >
        > > I'm certainly not an expert, but you might want to consider something
        > like
        > >
        > > sectionList
        > > : section (SECT section)*
        > > ;
        > > section!
        > > : (d:docsecfmtitems)? (h:hdrftritems)? p:paraitems
        > > { ## = #(SECT #(PARA #p) #(DOCSEC #d) #(HDRFTR #h)); }
        > > ;
        > >
        > > This would give you a fifo list of one or more SECT nodes, where each
        > SECT
        > > node has a PARA, DOCSEC and HDRFTR node entry.
        > >
        > > By the way, you can get confusing results when you build AST nodes with
        > > entries that can be either empty or a list. I've found that its usually
        > > easier to build a separate AST node for these, as with the PARA, DOCSEC
        > > and
        > > HDRFTR nodes above.
        > >
        > > Ken
        > > ken@...
        > >
        > > -----Original Message-----
        > > From: kevin.burton@... [mailto:kevin.burton@...]
        > > Sent: Tuesday, December 07, 1999 10:04 AM
        > > To: antlr-interest@onelist.com
        > > Subject: RE: [antlr-interest] AST Tree construction.
        > >
        > >
        > > From: kevin.burton@...
        > >
        > > I really do not want #(#(df), #(hf), #(p)). I agree that it does not
        > make
        > > sense to have a tree as a root node. Is what I want is to form three
        > > separate trees, one for df, one for hf, and one for p. For one I don't
        > > know
        > > how to form a tree given a list of input, and I don't know how to
        > > correctly
        > > handle the recursion that is implicit with the 'section' rule below. If
        > I
        > > need to create a special node as the name of the section that would be
        > > nice
        > > to, but I am not sure how to do that either. Maybe the grammar is
        > unclear.
        > > Maybe it should be something like:
        > >
        > > sectionList
        > > : section (SECT sectionList)?
        > > ;
        > >
        > > section
        > > : (docsecfmtitems)? (hdrftritems)? paraitems
        > > ;
        > >
        > > Would this be easier to generate a tree from? But, again for the section
        > > rule I would probably need to create a node that could act as the root.
        > >
        > > Thank you for you continued help.
        > >
        > > Kevin Burton
        > > Kevin.Burton@...
        > >
        > > > -----Original Message-----
        > > > From: parrt@... [SMTP:parrt@...]
        > > > Sent: Tuesday, December 07, 1999 11:52 AM
        > > > To: antlr-interest@onelist.com
        > > > Subject: RE: [antlr-interest] AST Tree construction.
        > > >
        > > > From: <parrt@...>
        > > >
        > > > kevin.burton@... writes:
        > > > > section!
        > > > > : ((df:docsecfmtitems)?
        > > > >(hf:hdrftritems)? p:paraitems) (SECT^ section)?
        > > > > { #section = #(#(df), #(hf), #(p));
        > > > >}
        > > >
        > > > This looks weird to me. Get rid of the extra parens. #(...) is a
        > > > tree. #label is a reference to a tree. Also, you use ^, but you have
        >
        > > > ! on the rule saying not to autogen the tree...not sure what would
        > > > happen. Use
        > > >
        > > > #section = #(#df, #hf, #p);
        > > >
        > > > Actually #df as root won't work probably as it's a tree not a single
        > > > node. That makes no sense to have a tree as a root.
        > > >
        > > >
        > > > Hope this helps.
        > > >
        > > > Ter
        > > >
        > > >
        > >
        > >
        >
      • Sinan
        ... My mailer has been dead for 3 days. Now I ll look ate what you have an send you response... Sinan
        Message 3 of 18 , Dec 8, 1999
        • 0 Attachment
          kevin.burton@... wrote:
          >


          My mailer has been dead for 3 days. Now I'll look ate what you have an
          send you response...


          Sinan
        Your message has been successfully submitted and would be delivered to recipients shortly.