Loading ...
Sorry, an error occurred while loading the content.
Skip to search.
 

RE: AST Tree construction.

Expand Messages
  • Ken Lidster
    Kevin, No problem. I do recommend, though, that you change sectionList ... ; to sectionList ... ; as it eliminates the recursion and does simple AST list
    Message 1 of 18 , Dec 7 2:13 PM
      Kevin,

      No problem. I do recommend, though, that you change

      sectionList
      : section (SECT sectionList)?
      ;
      to
      sectionList
      : section (SECT section)*
      ;

      as it eliminates the recursion and does simple AST list construction.

      As to context-sensitive tokens, there are a couple of options. The simplest,
      of course, is to redefine your grammar so that the nondeterminism is
      removed. For example, you said that there is a set of tokens that are used
      by both the 'paraitems' and 'docsetfmtitems' rules. Can you move these into
      a common rule that is referenced by both?

      The other option is the new 'Lexer States' mechanism that is available with
      2.6.0. I haven't used it myself, but the documentation (see "streams.html"
      in the doc directory) indicates this could be a promising solution to this
      class of problem.

      Ken
      ken@...

      -----Original Message-----
      From: kevin.burton@... [mailto:kevin.burton@...]
      Sent: Tuesday, December 07, 1999 1:35 PM
      To: antlr-interest@onelist.com
      Subject: RE: [antlr-interest] AST Tree construction.


      From: kevin.burton@...

      Ken,

      Thanks to your comments I was able to put together the following code that
      does mostly what I want.

      tokens {
      SECTION;
      DOCSEC;
      HDRFTR;
      PARA;
      }

      . . . . .

      sectionList
      : section (SECT sectionList)?
      ;

      section!
      : d:docsecfmtitems h:hdrftritems p:paraitems
      { #section = #([SECTION, "section"],
      ([DOCSEC, "docsec"], #d), ([HDRFTR, "hdrftr"], #h), ([PARA, "para"], #p) );
      }
      | d:docsecfmtitems p:paraitems
      { #section = #([SECTION, "section"],
      ([DOCSEC, "docsec"], #d), ([PARA, "para"], #p) ); }
      | h:hdrftritems p:paraitems
      { #section = #([SECTION, "section"],
      ([HDRFTR, "hdrftr"], #h), ([PARA, "para"], #p) ); }
      | p:paraitems
      { #section = #([SECTION, "section"], ([PARA,
      "para"], #p) ); }
      ;

      I get a "section" tree that contains at most three trees. This is a great
      step forward. Thank you.

      The problem is that doing this introduced a nondeterminism warning into my
      grammar. It turns out that 'paraitems' can contain tokens that are defined
      in 'docsecfmtitems'. Is there a mechanism that I can distinquish between
      tokens based on context? Any suggestions as to how I can resolve these
      nondeterminism warnings (errors)?

      Thanks again.

      Kevin

      > -----Original Message-----
      > From: kevin.burton@... [SMTP:kevin.burton@...]
      > Sent: Tuesday, December 07, 1999 1:51 PM
      > To: antlr-interest@onelist.com
      > Subject: RE: [antlr-interest] AST Tree construction.
      >
      > From: kevin.burton@...
      >
      > Ken,
      >
      > Thanks for your comments.
      >
      > Is this the correct syntax? I tried it and am getting "Syntax error in
      > action: Error: line(62), mismatched char: '#' expected ')'
      >
      > Kevin
      >
      > > -----Original Message-----
      > > From: Ken Lidster [SMTP:Ken.Lidster@...]
      > > Sent: Tuesday, December 07, 1999 1:18 PM
      > > To: 'antlr-interest@onelist.com'
      > > Subject: RE: [antlr-interest] AST Tree construction.
      > >
      > > From: Ken Lidster <Ken.Lidster@...>
      > >
      > > Kevin,
      > >
      > > I'm certainly not an expert, but you might want to consider something
      > like
      > >
      > > sectionList
      > > : section (SECT section)*
      > > ;
      > > section!
      > > : (d:docsecfmtitems)? (h:hdrftritems)? p:paraitems
      > > { ## = #(SECT #(PARA #p) #(DOCSEC #d) #(HDRFTR #h)); }
      > > ;
      > >
      > > This would give you a fifo list of one or more SECT nodes, where each
      > SECT
      > > node has a PARA, DOCSEC and HDRFTR node entry.
      > >
      > > By the way, you can get confusing results when you build AST nodes with
      > > entries that can be either empty or a list. I've found that its usually
      > > easier to build a separate AST node for these, as with the PARA, DOCSEC
      > > and
      > > HDRFTR nodes above.
      > >
      > > Ken
      > > ken@...
      > >
      > > -----Original Message-----
      > > From: kevin.burton@... [mailto:kevin.burton@...]
      > > Sent: Tuesday, December 07, 1999 10:04 AM
      > > To: antlr-interest@onelist.com
      > > Subject: RE: [antlr-interest] AST Tree construction.
      > >
      > >
      > > From: kevin.burton@...
      > >
      > > I really do not want #(#(df), #(hf), #(p)). I agree that it does not
      > make
      > > sense to have a tree as a root node. Is what I want is to form three
      > > separate trees, one for df, one for hf, and one for p. For one I don't
      > > know
      > > how to form a tree given a list of input, and I don't know how to
      > > correctly
      > > handle the recursion that is implicit with the 'section' rule below. If
      > I
      > > need to create a special node as the name of the section that would be
      > > nice
      > > to, but I am not sure how to do that either. Maybe the grammar is
      > unclear.
      > > Maybe it should be something like:
      > >
      > > sectionList
      > > : section (SECT sectionList)?
      > > ;
      > >
      > > section
      > > : (docsecfmtitems)? (hdrftritems)? paraitems
      > > ;
      > >
      > > Would this be easier to generate a tree from? But, again for the section
      > > rule I would probably need to create a node that could act as the root.
      > >
      > > Thank you for you continued help.
      > >
      > > Kevin Burton
      > > Kevin.Burton@...
      > >
      > > > -----Original Message-----
      > > > From: parrt@... [SMTP:parrt@...]
      > > > Sent: Tuesday, December 07, 1999 11:52 AM
      > > > To: antlr-interest@onelist.com
      > > > Subject: RE: [antlr-interest] AST Tree construction.
      > > >
      > > > From: <parrt@...>
      > > >
      > > > kevin.burton@... writes:
      > > > > section!
      > > > > : ((df:docsecfmtitems)?
      > > > >(hf:hdrftritems)? p:paraitems) (SECT^ section)?
      > > > > { #section = #(#(df), #(hf), #(p));
      > > > >}
      > > >
      > > > This looks weird to me. Get rid of the extra parens. #(...) is a
      > > > tree. #label is a reference to a tree. Also, you use ^, but you have
      >
      > > > ! on the rule saying not to autogen the tree...not sure what would
      > > > happen. Use
      > > >
      > > > #section = #(#df, #hf, #p);
      > > >
      > > > Actually #df as root won't work probably as it's a tree not a single
      > > > node. That makes no sense to have a tree as a root.
      > > >
      > > >
      > > > Hope this helps.
      > > >
      > > > Ter
      > > >
      > > >
      > >
      > >
      >
    • Sinan
      ... My mailer has been dead for 3 days. Now I ll look ate what you have an send you response... Sinan
      Message 2 of 18 , Dec 8 3:00 PM
        kevin.burton@... wrote:
        >


        My mailer has been dead for 3 days. Now I'll look ate what you have an
        send you response...


        Sinan
      Your message has been successfully submitted and would be delivered to recipients shortly.