Loading ...
Sorry, an error occurred while loading the content.

[Cheetahtemplate-discuss] Decoding error

Expand Messages
  • erez dickman
    I ve recently changed the machine on which I run my webserver on, and depsite using the exact same code I get the following error message: Traceback (most
    Message 1 of 3 , Jul 30 6:42 AM
    • 0 Attachment
      I've recently changed the machine on which I run my webserver on, and depsite using the exact same code I get the following error message:
      Traceback (most recent call last):
        File "/usr/local/lib/python2.4/site-packages/cherrypy/_cphttptools.py", line 105, in _run
          self.main()
        File "/usr/local/lib/python2.4/site-packages/cherrypy/_cphttptools.py", line 254, in main
          body = page_handler(*virtual_path, **self.params)
        File "UserServer.py", line 113, in index
          return str( page )
        File "/usr/local/lib/python2.4/site-packages/Cheetah/Template.py", line 990, in __str__
          def __str__(self): return getattr(self, mainMethName)()
        File "../webserver/wholetemplates/page.py", line 136, in respond
          if _v is not None: write(_filter(_v, rawExpr='$header')) # from line 26, col 2.
        File "/usr/local/lib/python2.4/site-packages/Cheetah/Filters.py", line 142, in filter
          filtered = str(val)
        File "/usr/local/lib/python2.4/site-packages/Cheetah/Template.py", line 990, in __str__
          def __str__(self): return getattr(self, mainMethName)()
        File "../webserver/parttemplates/header.py", line 144, in respond
          return _dummyTrans and trans.response().getvalue() or ""
        File "/usr/local/lib/python2.4/site-packages/Cheetah/DummyTransaction.py", line 32, in getvalue
          return ''.join(outputChunks)
      UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 1953: ordinal not in range(128)

      Any idea on what might be causing the error?
    • Mike Orr
      ... You are probably using a different version of Cheetah. This was changed from StringIO to a list join, and list join is more strict about what types it
      Message 2 of 3 , Aug 4, 2006
      • 0 Attachment
        On 7/30/06, erez dickman <erez.dickman@...> wrote:
        > I've recently changed the machine on which I run my webserver on, and
        > depsite using the exact same code I get the following error message:

        > "/usr/local/lib/python2.4/site-packages/Cheetah/DummyTransaction.py",
        > line 32, in getvalue
        > return ''.join(outputChunks)
        > UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 1953:
        > ordinal not in range(128)
        >
        > Any idea on what might be causing the error?

        You are probably using a different version of Cheetah. This was
        changed from StringIO to a list join, and list join is more strict
        about what types it accepts. Recompile all your templates and see if
        the problem persists. All versions of Cheetah since 1.0rc1 require
        recompiling templates when changing Cheetah versions, as noted in the
        CHANGES file. If that still doesn't fix it...

        The underlying problem is exactly what it says: there's a non-ASCII
        character somewhere. Because it says UnicodeDecodeError instead of
        UnicodeEncodeError, I assume the value is in a unicode object and you
        have called str(t) to fill the template. This will not work due to a
        bug in str(): it blows up with non-ASCII strings, and the obvious
        workaround of calling sys.setdefaultencoding() does not work. A
        future version of Python will allow str() to return unicode, but for
        now we have to work around it.

        The easist workaround is to call t.respond() instead of str(t).
        unicode(t) may work but I think I still had problems with it.

        So we've dealt with the problem of an non-ASCII character in a unicode
        object. But the reverse problem also happens: a non-ASCII character
        in a str object. If any part of the template is unicode (either the
        text or a placeholder value), Cheetah promotes all strings to unicode
        and returns a unicode object. This fails for the same reason: Python
        doesn't know what charset the string is in, so it calls unicode(v),
        which is the same as unicode(v, 'ascii', 'strict'), and you get a
        UnicodeEncodeError.

        The workaround for this is to manually convert the offending value to
        unicode before Cheetah sees it. That way you can specify the
        character set and error-handling mechanism. If you're dealing with
        multilingual characters, it's best to do this immediately on input,
        and then you can assume all strings are unicode. You'll have to
        convert them back for output. Something like this may be useful:

        f,write( t.respond().encode('latin1', 'xmlcharrefreplace') )

        I can't remember exactly what incantation I used, but I had to use one
        for TurboCheetah, and another one in a filter. I can look them up if
        necessary.

        --
        Mike Orr <sluggoster@...>

        -------------------------------------------------------------------------
        Take Surveys. Earn Cash. Influence the Future of IT
        Join SourceForge.net's Techsay panel and you'll get the chance to share your
        opinions on IT & business topics through brief surveys -- and earn cash
        http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
        _______________________________________________
        Cheetahtemplate-discuss mailing list
        Cheetahtemplate-discuss@...
        https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss
      • Mike Orr
        ... I ve seen this happen when upgrading from MySQL 4 (which is not charset aware) to MySQL 5 (which has charsets), when pasting text from Word/Windows (which
        Message 3 of 3 , Aug 4, 2006
        • 0 Attachment
          On 8/4/06, Mike Orr <sluggoster@...> wrote:
          > On 7/30/06, erez dickman <erez.dickman@...> wrote:
          > > I've recently changed the machine on which I run my webserver on, and
          > > depsite using the exact same code I get the following error message:
          >
          > > "/usr/local/lib/python2.4/site-packages/Cheetah/DummyTransaction.py",
          > > line 32, in getvalue
          > > return ''.join(outputChunks)
          > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 1953:
          > > ordinal not in range(128)
          > >
          > > Any idea on what might be causing the error?
          >
          > You are probably using a different version of Cheetah. This was
          > changed from StringIO to a list join, and list join is more strict
          > about what types it accepts. Recompile all your templates and see if
          > the problem persists. All versions of Cheetah since 1.0rc1 require
          > recompiling templates when changing Cheetah versions, as noted in the
          > CHANGES file. If that still doesn't fix it...
          >
          > The underlying problem is exactly what it says: there's a non-ASCII
          > character somewhere. Because it says UnicodeDecodeError instead of
          > UnicodeEncodeError, I assume the value is in a unicode object and you
          > have called str(t) to fill the template. This will not work due to a
          > bug in str(): it blows up with non-ASCII strings, and the obvious
          > workaround of calling sys.setdefaultencoding() does not work. A
          > future version of Python will allow str() to return unicode, but for
          > now we have to work around it.
          >
          > The easist workaround is to call t.respond() instead of str(t).
          > unicode(t) may work but I think I still had problems with it.
          >
          > So we've dealt with the problem of an non-ASCII character in a unicode
          > object. But the reverse problem also happens: a non-ASCII character
          > in a str object.

          I've seen this happen when upgrading from MySQL 4 (which is not
          charset aware) to MySQL 5 (which has charsets), when pasting text from
          Word/Windows (which uses nonstandard characters like curly quotes), or
          Filemaker/Mac (which uses the MacRoman charset), or when reading
          something pasted into an HTML form (which is who the hell knows what
          charset, possibly misinterpreted one way by the browser and a
          different way by the server).

          --
          Mike Orr <sluggoster@...>

          -------------------------------------------------------------------------
          Take Surveys. Earn Cash. Influence the Future of IT
          Join SourceForge.net's Techsay panel and you'll get the chance to share your
          opinions on IT & business topics through brief surveys -- and earn cash
          http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
          _______________________________________________
          Cheetahtemplate-discuss mailing list
          Cheetahtemplate-discuss@...
          https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss
        Your message has been successfully submitted and would be delivered to recipients shortly.