Loading ...
Sorry, an error occurred while loading the content.
 

Re: Case insensitivity and the tolower function

Expand Messages
  • Ric Klaren
    Hi, ... Depends if this code is still in the next version for wide character support all this stuff needs to be changed anyway. EOF_CHAR should really be
    Message 1 of 3 , Nov 30, 2000
      Hi,

      On Wed, Nov 29, 2000 at 11:20:06AM -0800, John Lambert wrote:
      > I have been having more problems with the tolower function and it's handling
      > of EOF_CHAR.
      >
      > It seems that STL port will truncate the -1 down to 0xff :-(
      >
      > So I propose that EOF is not passed through the tolower function by changing
      > the function:
      >
      > int CharScanner::toLower(int c) const
      > {
      > return (c == EOF_CHAR ? EOF_CHAR : tolower(c));
      > }
      >
      > Ric, is this a reasonable change to include in the next version?

      Depends if this code is still in the next version for wide character
      support all this stuff needs to be changed anyway. EOF_CHAR should really
      be something like char_traits<char>::eof() (or char_traits<wchar>::eof()
      for wide characters). But this probably breaks a lot of compilers/support
      libraries. Btw could you check if changing EOF_CHAR to
      char_traits<char>::eof() fixes this? Or as a longshot shouldn't that be
      std::tolower(c)....

      Anyway for now I'll check this workaround into my devel tree.

      The new version will be up in a few minutes at:

      http://wwwhome.cs.utwente.nl/~klaren/antlr/antlr-devel.tar.gz

      Cheers,

      Ric


      >
      > Thanks,
      >
      > John
      >
      >
      >
      >
      >

      --
      -----+++++*****************************************************+++++++++-------
      ---- Ric Klaren ----- klaren@... ----- +31 4893722 ----
      -----+++++*****************************************************+++++++++-------
      Why don't we just invite them to dinner and massacre them all when they're
      drunk? You heard the man. There's seven hundred thousand of them.
      Ah? ... So it'd have to be something simple with pasta, then.
      --- From: Interesting Times by Terry Pratchet
      -----+++++*****************************************************+++++++++-------
    • Alexander Lenski
      Hi, I am a little bit confused with the CharScaner::consume() implementation (the code from the CharScanner is shown below). Anyway (on true == caseSensitive
      Message 2 of 3 , Dec 3, 2000
        Hi,
        I am a little bit confused with the CharScaner::consume() implementation (the code from the CharScanner is
        shown below). Anyway (on true == caseSensitive or on false == caseSensitive) the inputState->getInput().LA(1)
        is called to get a char (see the CharScaner's  consume() and LA() implementation below).
        Why is part under the question not the (to keep the caseSensitive in use):
        // This is a start of the ??? 
                  int c = LA(1);
                  append(c);
        // This is the end
        Or why is it not the (to do the same just with better performance):
        // This is a start of the ??? 
                  int c = inputState->getInput().LA(1);
                  append(c);
        // This is the end
        Original code:
         
        void CharScanner::consume()
        {    if (inputState->guessing == 0)
             {
        // This is a start of the ??? 
                  int c = LA(1);
                  if (caseSensitive)
                  {   append(c);
                  }
                  else
                  {   // use input.LA(), not LA(), to get original case
                       // CharScanner.LA() would toLower it.
                       append(inputState->getInput().LA(1));
                  }
        // This is the end
                  if (c == '\t')
                  {    tab();
                  }
                  else
                  {    inputState->column++;
                  }
             }
             inputState->getInput().consume();
        }
        inline int CharScanner::LA(int i)
        {    if ( caseSensitive )
             {    return inputState->getInput().LA(i);
             }
             else
             {    return toLower(inputState->getInput().LA(i));
             }
        }
        Thanks,
        Alexander Lenski
         
        ----- Original Message -----
        From: "Ric Klaren" <klaren@...>
        Sent: Thursday, November 30, 2000 4:17 AM
        Subject: [antlr-interest] Re: Case insensitivity and the tolower function

        > Hi,
        >
        > On Wed, Nov 29,
        2000 at 11:20:06AM -0800, John Lambert wrote:
        > > I have been having
        more problems with the tolower function and it's handling
        > > of
        EOF_CHAR.
        > >
        > > It seems that STL port will truncate the -1
        down to 0xff :-(
        > >
        > > So I propose that EOF is not passed
        through the tolower function by changing
        > > the function:
        > >
        > > int CharScanner::toLower(int c) const
        > > {
        > >
        return (c == EOF_CHAR ? EOF_CHAR : tolower(c));
        > > }
        > >
        > > Ric, is this a reasonable change to include in the next
        version?
        >
        > Depends if this code is still in the next version for
        wide character
        > support all this stuff needs to be changed anyway.
        EOF_CHAR should really
        > be something like char_traits<char>::eof()
        (or char_traits<wchar>::eof()
        > for wide characters). But this
        probably breaks a lot of compilers/support
        > libraries. Btw could you
        check if changing EOF_CHAR to
        > char_traits<char>::eof() fixes this?
        Or as a longshot shouldn't that be
        > std::tolower(c)....
        >
        >
        Anyway for now I'll check this workaround into my devel tree.
        >
        >
        The new version will be up in a few minutes at:
        >
        >
        href="http://wwwhome.cs.utwente.nl/~klaren/antlr/antlr-devel.tar.gz">http://wwwhome.cs.utwente.nl/~klaren/antlr/antlr-devel.tar.gz
        >
        > Cheers,
        >
        > Ric
        >
        >
        > >
        > > Thanks,
        > >
        > > John
        > >
        > >
        > >
        > >
        > >
        >
        > --
        >
        -----+++++*****************************************************+++++++++-------
        >
              ---- Ric Klaren -----
        klaren@... ----- +31 4893722  ----
        >
        -----+++++*****************************************************+++++++++-------
        >
          Why don't we just invite them to dinner and massacre them all when they're
        >   drunk? You heard the man. There's seven hundred thousand
        of them.
        >   Ah? ... So it'd have to be something simple with pasta,
        then.
        >
                                      --- From: Interesting Times by Terry Pratchet
        >
        -----+++++*****************************************************+++++++++-------
        >
        > -------------------------- eGroups Sponsor
        -------------------------~-~>
        > eLerts
        > It's Easy. It's Fun.
        Best of All, it's Free!
        >
        href="http://click.egroups.com/1/9699/1/_/492272/_/975575859/">http://click.egroups.com/1/9699/1/_/492272/_/975575859/
        > ---------------------------------------------------------------------_->
        >
        >
        >
        >
      Your message has been successfully submitted and would be delivered to recipients shortly.