Loading ...
Sorry, an error occurred while loading the content.

[Cheetahtemplate-discuss] Unicode strings as arguments to placeholder functions/methods

Expand Messages
  • Stefan Schwarzer
    Hello, ... # encoding: latin1 from Cheetah import Template def f(arg): return arg namespace = { f : f} t = Template.Template(u $f(u xüy ) ,
    Message 1 of 4 , Apr 2, 2007
    • 0 Attachment
      Hello,

      I have a file test.py with this content:

      -----
      # encoding: latin1

      from Cheetah import Template

      def f(arg):
      return arg

      namespace = {'f': f}

      t = Template.Template(u'$f(u"xüy")', namespaces=[namespace])

      print t.respond().encode("latin1")
      -----

      ("ü" is u umlaut.) According to Vim, the "fileencoding" is
      actually latin1.

      The file's output is

      -----
      Traceback (most recent call last):
      File "cheetah_bug.py", line 10, in <module>
      t = Template.Template(u"""$f(u"xüy")""", namespaces=[namespace])
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py", line 1200, in __init__
      self._compile(source, file, compilerSettings=compilerSettings)
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py", line 1488, in _compile
      keepRefToGeneratedCode=True)
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py", line 717, in compile
      generatedModuleCode = compiler.getModuleCode()
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1858, in getModuleCode
      return self.wrapModuleDef()
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1912, in wrapModuleDef
      'classes':self.classDefs(),
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1961, in classDefs
      classDefs = [str(klass) for klass in self._finishedClasses() ]
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1431, in classDef
      return self.wrapClassDef()
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1455, in wrapClassDef
      addMethods()
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1445, in addMethods
      self.methodDefs(),
      File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1480, in methodDefs
      methodDefs = [str(methGen) for methGen in self._finishedMethods() ]
      UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 665: ordinal not in range(128)
      -----

      I expected the output to be

      -----
      xüy
      -----

      on a terminal with latin1 encoding.

      The above execution is with

      - Python 2.5 on Gentoo Linux

      - CheetahTemplate 2.0rc7, installed with easy_install,
      but with revision 1.150 of Compile.py from
      http://cheetahtemplate.cvs.sourceforge.net/*checkout*/cheetahtemplate/Cheetah/src/Compiler.py?revision=1.150
      (to deal with another bug regarding multiple newlines at the
      start or end of the string)

      Am I doing something wrong? If yes, what should I do instead?

      Stefan

      -------------------------------------------------------------------------
      Take Surveys. Earn Cash. Influence the Future of IT
      Join SourceForge.net's Techsay panel and you'll get the chance to share your
      opinions on IT & business topics through brief surveys-and earn cash
      http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
      _______________________________________________
      Cheetahtemplate-discuss mailing list
      Cheetahtemplate-discuss@...
      https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss
    • Tavis Rudd
      Hi Stefan, It works for me if the template is an encoded byte string instead of a unicode string, but it *should* work with both. I m getting the same error
      Message 2 of 4 , Apr 2, 2007
      • 0 Attachment
        Hi Stefan,
        It works for me if the template is an encoded byte string instead of a
        unicode string, but it *should* work with both. I'm getting the same
        error with the cvs version of Cheetah so there's definitely a bug in the
        compiler.

        assert '$f(u"xüy")'==u'$f(u"xüy")'.encode('latin-1')
        passes in pure Python.

        I'm looking into it now.

        Tavis

        On Tue, 3 Apr 2007, Stefan Schwarzer wrote:

        > Hello,
        >
        > I have a file test.py with this content:
        >
        > -----
        > # encoding: latin1
        >
        > from Cheetah import Template
        >
        > def f(arg):
        > return arg
        >
        > namespace = {'f': f}
        >
        > t = Template.Template(u'$f(u"xüy")', namespaces=[namespace])
        >
        > print t.respond().encode("latin1")
        > -----
        >
        > ("ü" is u umlaut.) According to Vim, the "fileencoding" is
        > actually latin1.
        >
        > The file's output is
        >
        > -----
        > Traceback (most recent call last):
        > File "cheetah_bug.py", line 10, in <module>
        > t = Template.Template(u"""$f(u"xüy")""", namespaces=[namespace])
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py", line 1200, in __init__
        > self._compile(source, file, compilerSettings=compilerSettings)
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py", line 1488, in _compile
        > keepRefToGeneratedCode=True)
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py", line 717, in compile
        > generatedModuleCode = compiler.getModuleCode()
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1858, in getModuleCode
        > return self.wrapModuleDef()
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1912, in wrapModuleDef
        > 'classes':self.classDefs(),
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1961, in classDefs
        > classDefs = [str(klass) for klass in self._finishedClasses() ]
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1431, in classDef
        > return self.wrapClassDef()
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1455, in wrapClassDef
        > addMethods()
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1445, in addMethods
        > self.methodDefs(),
        > File "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py", line 1480, in methodDefs
        > methodDefs = [str(methGen) for methGen in self._finishedMethods() ]
        > UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 665: ordinal not in range(128)
        > -----
        >
        > I expected the output to be
        >
        > -----
        > xüy
        > -----
        >
        > on a terminal with latin1 encoding.
        >
        > The above execution is with
        >
        > - Python 2.5 on Gentoo Linux
        >
        > - CheetahTemplate 2.0rc7, installed with easy_install,
        > but with revision 1.150 of Compile.py from
        > http://cheetahtemplate.cvs.sourceforge.net/*checkout*/cheetahtemplate/Cheetah/src/Compiler.py?revision=1.150
        > (to deal with another bug regarding multiple newlines at the
        > start or end of the string)
        >
        > Am I doing something wrong? If yes, what should I do instead?
        >
        > Stefan
        >
        > -------------------------------------------------------------------------
        > Take Surveys. Earn Cash. Influence the Future of IT
        > Join SourceForge.net's Techsay panel and you'll get the chance to share your
        > opinions on IT & business topics through brief surveys-and earn cash
        > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
        > _______________________________________________
        > Cheetahtemplate-discuss mailing list
        > Cheetahtemplate-discuss@...
        > https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss
        >
      • Tavis Rudd
        Btw, this works fine if you stick the template source in an external .tmpl file: #### foo.tmpl #encoding: latin-1 $f(u xüy ) #### test.py from Cheetah import
        Message 3 of 4 , Apr 2, 2007
        • 0 Attachment
          Btw, this works fine if you stick the template source in an external .tmpl
          file:

          #### foo.tmpl
          #encoding: latin-1
          $f(u"xüy")

          #### test.py
          from Cheetah import Template

          def f(arg):
          return arg

          namespace = {'f': f}

          t = Template.Template(file='/tmp/test_tmpl3.tmpl', namespaces=[namespace])
          print t.respond().encode("latin1")

          The bug only affects the use of unicode strings as the source argument.

          Tavis

          On Mon, 2 Apr 2007, Tavis Rudd wrote:

          > Hi Stefan,
          > It works for me if the template is an encoded byte string instead of a
          > unicode string, but it *should* work with both. I'm getting the same error
          > with the cvs version of Cheetah so there's definitely a bug in the compiler.
          >
          > assert '$f(u"xüy")'==u'$f(u"xüy")'.encode('latin-1')
          > passes in pure Python.
          >
          > I'm looking into it now.
          >
          > Tavis
          >
          > On Tue, 3 Apr 2007, Stefan Schwarzer wrote:
          >
          >> Hello,
          >>
          >> I have a file test.py with this content:
          >>
          >> -----
          >> # encoding: latin1
          >>
          >> from Cheetah import Template
          >>
          >> def f(arg):
          >> return arg
          >>
          >> namespace = {'f': f}
          >>
          >> t = Template.Template(u'$f(u"xüy")', namespaces=[namespace])
          >>
          >> print t.respond().encode("latin1")
          >> -----
          >>
          >> ("ü" is u umlaut.) According to Vim, the "fileencoding" is
          >> actually latin1.
          >>
          >> The file's output is
          >>
          >> -----
          >> Traceback (most recent call last):
          >> File "cheetah_bug.py", line 10, in <module>
          >> t = Template.Template(u"""$f(u"xüy")""", namespaces=[namespace])
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py",
          >> line 1200, in __init__
          >> self._compile(source, file, compilerSettings=compilerSettings)
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py",
          >> line 1488, in _compile
          >> keepRefToGeneratedCode=True)
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py",
          >> line 717, in compile
          >> generatedModuleCode = compiler.getModuleCode()
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
          >> line 1858, in getModuleCode
          >> return self.wrapModuleDef()
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
          >> line 1912, in wrapModuleDef
          >> 'classes':self.classDefs(),
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
          >> line 1961, in classDefs
          >> classDefs = [str(klass) for klass in self._finishedClasses() ]
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
          >> line 1431, in classDef
          >> return self.wrapClassDef()
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
          >> line 1455, in wrapClassDef
          >> addMethods()
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
          >> line 1445, in addMethods
          >> self.methodDefs(),
          >> File
          >> "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
          >> line 1480, in methodDefs
          >> methodDefs = [str(methGen) for methGen in self._finishedMethods() ]
          >> UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in
          >> position 665: ordinal not in range(128)
          >> -----
          >>
          >> I expected the output to be
          >>
          >> -----
          >> xüy
          >> -----
          >>
          >> on a terminal with latin1 encoding.
          >>
          >> The above execution is with
          >>
          >> - Python 2.5 on Gentoo Linux
          >>
          >> - CheetahTemplate 2.0rc7, installed with easy_install,
          >> but with revision 1.150 of Compile.py from
          >> http://cheetahtemplate.cvs.sourceforge.net/*checkout*/cheetahtemplate/Cheetah/src/Compiler.py?revision=1.150
          >> (to deal with another bug regarding multiple newlines at the
          >> start or end of the string)
          >>
          >> Am I doing something wrong? If yes, what should I do instead?
          >>
          >> Stefan
          >>
          >> -------------------------------------------------------------------------
          >> Take Surveys. Earn Cash. Influence the Future of IT
          >> Join SourceForge.net's Techsay panel and you'll get the chance to share
          >> your
          >> opinions on IT & business topics through brief surveys-and earn cash
          >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
          >> _______________________________________________
          >> Cheetahtemplate-discuss mailing list
          >> Cheetahtemplate-discuss@...
          >> https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss
          >>
          >
        • Tavis Rudd
          It s fixed in the cvs now. The compiler was coercing some things to str where it shouldn t have. ... Take Surveys. Earn Cash. Influence the Future of IT Join
          Message 4 of 4 , Apr 2, 2007
          • 0 Attachment
            It's fixed in the cvs now. The compiler was coercing some things to str
            where it shouldn't have.

            On Mon, 2 Apr 2007, Tavis Rudd wrote:

            > Btw, this works fine if you stick the template source in an external .tmpl
            > file:
            >
            > #### foo.tmpl
            > #encoding: latin-1
            > $f(u"xüy")
            >
            > #### test.py
            > from Cheetah import Template
            >
            > def f(arg):
            > return arg
            >
            > namespace = {'f': f}
            >
            > t = Template.Template(file='/tmp/test_tmpl3.tmpl', namespaces=[namespace])
            > print t.respond().encode("latin1")
            >
            > The bug only affects the use of unicode strings as the source argument.
            >
            > Tavis
            >
            > On Mon, 2 Apr 2007, Tavis Rudd wrote:
            >
            >> Hi Stefan,
            >> It works for me if the template is an encoded byte string instead of a
            >> unicode string, but it *should* work with both. I'm getting the same
            >> error with the cvs version of Cheetah so there's definitely a bug in the
            >> compiler.
            >>
            >> assert '$f(u"xüy")'==u'$f(u"xüy")'.encode('latin-1')
            >> passes in pure Python.
            >>
            >> I'm looking into it now.
            >>
            >> Tavis
            >>
            >> On Tue, 3 Apr 2007, Stefan Schwarzer wrote:
            >>
            >> > Hello,
            >> >
            >> > I have a file test.py with this content:
            >> >
            >> > -----
            >> > # encoding: latin1
            >> >
            >> > from Cheetah import Template
            >> >
            >> > def f(arg):
            >> > return arg
            >> >
            >> > namespace = {'f': f}
            >> >
            >> > t = Template.Template(u'$f(u"xüy")', namespaces=[namespace])
            >> >
            >> > print t.respond().encode("latin1")
            >> > -----
            >> >
            >> > ("ü" is u umlaut.) According to Vim, the "fileencoding" is
            >> > actually latin1.
            >> >
            >> > The file's output is
            >> >
            >> > -----
            >> > Traceback (most recent call last):
            >> > File "cheetah_bug.py", line 10, in <module>
            >> > t = Template.Template(u"""$f(u"xüy")""", namespaces=[namespace])
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py",
            >> > line 1200, in __init__
            >> > self._compile(source, file, compilerSettings=compilerSettings)
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py",
            >> > line 1488, in _compile
            >> > keepRefToGeneratedCode=True)
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Template.py",
            >> > line 717, in compile
            >> > generatedModuleCode = compiler.getModuleCode()
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
            >> > line 1858, in getModuleCode
            >> > return self.wrapModuleDef()
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
            >> > line 1912, in wrapModuleDef
            >> > 'classes':self.classDefs(),
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
            >> > line 1961, in classDefs
            >> > classDefs = [str(klass) for klass in self._finishedClasses() ]
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
            >> > line 1431, in classDef
            >> > return self.wrapClassDef()
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
            >> > line 1455, in wrapClassDef
            >> > addMethods()
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
            >> > line 1445, in addMethods
            >> > self.methodDefs(),
            >> > File
            >> > "/usr/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg/Cheetah/Compiler.py",
            >> > line 1480, in methodDefs
            >> > methodDefs = [str(methGen) for methGen in self._finishedMethods() ]
            >> > UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in
            >> > position 665: ordinal not in range(128)
            >> > -----
            >> >
            >> > I expected the output to be
            >> >
            >> > -----
            >> > xüy
            >> > -----
            >> >
            >> > on a terminal with latin1 encoding.
            >> >
            >> > The above execution is with
            >> >
            >> > - Python 2.5 on Gentoo Linux
            >> >
            >> > - CheetahTemplate 2.0rc7, installed with easy_install,
            >> > but with revision 1.150 of Compile.py from
            >> > http://cheetahtemplate.cvs.sourceforge.net/*checkout*/cheetahtemplate/Cheetah/src/Compiler.py?revision=1.150
            >> > (to deal with another bug regarding multiple newlines at the
            >> > start or end of the string)
            >> >
            >> > Am I doing something wrong? If yes, what should I do instead?
            >> >
            >> > Stefan
            >> >
            >> > -------------------------------------------------------------------------
            >> > Take Surveys. Earn Cash. Influence the Future of IT
            >> > Join SourceForge.net's Techsay panel and you'll get the chance to share
            >> > your
            >> > opinions on IT & business topics through brief surveys-and earn cash
            >> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
            >> > _______________________________________________
            >> > Cheetahtemplate-discuss mailing list
            >> > Cheetahtemplate-discuss@...
            >> > https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss
            >> >
            >>
            >
          Your message has been successfully submitted and would be delivered to recipients shortly.