Loading ...
Sorry, an error occurred while loading the content.

python3 and multibyte strings

Expand Messages
  • Stanislav Kondratyev
    Hello! I’m reposting this issue from vim_multibyte thanks to John Beckett s kind advice. I was experimenting with the python3 interface and I stumbled upon
    Message 1 of 6 , May 5, 2012
    • 0 Attachment
      Hello!

      I’m reposting this issue from vim_multibyte thanks to John Beckett's
      kind advice.

      I was experimenting with the python3 interface and I stumbled upon the
      following phenomenon. Suppose the cursor is in the string
      абвгдеёжзийклмнопрстуфхцчшщъыьэюя
      in a utf-8 encoded file. I run the following commands:
      :py3 import vim
      :py3 vim.current.line = vim.current.line
      Surprisingly last 16½ of 33 characters disappear.

      The same effect can be produced e. g. with
      :py3 l = vim.current.line
      :py3 vim.current.line = l
      However, the string l can be correctly printed and sent to vim via
      vim.command and setline. Hence I conclude that the transition from vim
      to python does not cause any loss of information. On the contrary,
      when the string is being passed from python to vim, vim (or python?)
      calculates that it is 33 ‘units’ long and copies 33 *bytes*.

      Experiments with less appaulling accented Latin letters show the same
      effect. For example,
      when the command
      :py3 vim.current.line = vim.current.line
      is applied to the line ‘aàáâãä', it yields
      aàá<c3>

      I’m running vim 7.3 on Windows 7.

      I’d like to find out whether this is a common effect and whether there
      exists an elegant way of changing multibyte strings from python3.

      Any help is appreciated. :)

      Stanislav

      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • lilydjwg
      ... I can t reproduce it on Windows XP with cp936 and Linux with UTF-8 locale. What s the value of your encoding option and the exact version of your Vim? --
      Message 2 of 6 , May 6, 2012
      • 0 Attachment
        On Sat, May 05, 2012 at 03:25:28AM -0700, Stanislav Kondratyev wrote:
        > Hello!
        >
        > I’m reposting this issue from vim_multibyte thanks to John Beckett's
        > kind advice.
        >
        > I was experimenting with the python3 interface and I stumbled upon the
        > following phenomenon. Suppose the cursor is in the string
        > абвгдеёжзийклмнопрстуфхцчшщъыьэюя
        > in a utf-8 encoded file. I run the following commands:
        > :py3 import vim
        > :py3 vim.current.line = vim.current.line
        > Surprisingly last 16½ of 33 characters disappear.
        >
        > The same effect can be produced e. g. with
        > :py3 l = vim.current.line
        > :py3 vim.current.line = l
        > However, the string l can be correctly printed and sent to vim via
        > vim.command and setline. Hence I conclude that the transition from vim
        > to python does not cause any loss of information. On the contrary,
        > when the string is being passed from python to vim, vim (or python?)
        > calculates that it is 33 ‘units’ long and copies 33 *bytes*.
        >
        > Experiments with less appaulling accented Latin letters show the same
        > effect. For example,
        > when the command
        > :py3 vim.current.line = vim.current.line
        > is applied to the line ‘aàáâãä', it yields
        > aàá<c3>
        >
        > I’m running vim 7.3 on Windows 7.
        >
        > I’d like to find out whether this is a common effect and whether there
        > exists an elegant way of changing multibyte strings from python3.
        >
        > Any help is appreciated. :)
        >
        > Stanislav

        I can't reproduce it on Windows XP with cp936 and Linux with UTF-8 locale.
        What's the value of your 'encoding' option and the exact version of your
        Vim?

        --
        Best regards,
        lilydjwg

        Linux Vim Python 我的博客:
        http://lilydjwg.is-programmer.com/
        --
        A: Because it obfuscates the reading.
        Q: Why is top posting so bad?

        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • Stanislav Kondratyev
        ... My Vim is VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Oct 27 2010 17:59:02) MS-Windows 32-bit GUI version with OLE support Included patches: 1-46 Compiled
        Message 3 of 6 , May 7, 2012
        • 0 Attachment
          On 6 май, 15:19, lilydjwg <lilyd...@...> wrote:
          > I can't reproduce it on Windows XP with cp936 and Linux with UTF-8 locale.
          > What's the value of your 'encoding' option and the exact version of your
          > Vim?
          >
          > --
          > Best regards,
          > lilydjwg
          >
          > Linux Vim Python 我的博客:http://lilydjwg.is-programmer.com/
          > --
          > A: Because it obfuscates the reading.
          > Q: Why is top posting so bad?

          My Vim is

          VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Oct 27 2010 17:59:02)
          MS-Windows 32-bit GUI version with OLE support
          Included patches: 1-46
          Compiled by Bram@KIBAALE
          Big version with GUI.

          I’ve set the encoding to utf-8 in the vimrc.

          Sorry, is cp936 a multibyte encoding?

          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        • lilydjwg
          ... Yes. I think you need to get a newer Vim. This was fixed around patch number 200. -- Best regards, lilydjwg Linux Vim Python §Úªº³Õ«È:
          Message 4 of 6 , May 7, 2012
          • 0 Attachment
            On Mon, May 07, 2012 at 01:36:44PM -0700, Stanislav Kondratyev wrote:
            > My Vim is
            >
            > VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Oct 27 2010 17:59:02)
            > MS-Windows 32-bit GUI version with OLE support
            > Included patches: 1-46
            > Compiled by Bram@KIBAALE
            > Big version with GUI.
            >
            > I’ve set the encoding to utf-8 in the vimrc.
            >
            > Sorry, is cp936 a multibyte encoding?

            Yes.

            I think you need to get a newer Vim. This was fixed around patch number 200.

            --
            Best regards,
            lilydjwg

            Linux Vim Python 我的博客:
            http://lilydjwg.is-programmer.com/
            --
            A: Because it obfuscates the reading.
            Q: Why is top posting so bad?

            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
          • Tony Mechelynck
            ... FYI, the latest source is 7.3.515. Best regards, Tony. -- Cigarette, n.: A fire at one end, a fool at the other, and a bit of tobacco in between. -- You
            Message 5 of 6 , May 7, 2012
            • 0 Attachment
              On 08/05/12 07:13, lilydjwg wrote:
              > On Mon, May 07, 2012 at 01:36:44PM -0700, Stanislav Kondratyev wrote:
              >> My Vim is
              >>
              >> VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Oct 27 2010 17:59:02)
              >> MS-Windows 32-bit GUI version with OLE support
              >> Included patches: 1-46
              >> Compiled by Bram@KIBAALE
              >> Big version with GUI.
              >>
              >> I’ve set the encoding to utf-8 in the vimrc.
              >>
              >> Sorry, is cp936 a multibyte encoding?
              >
              > Yes.
              >
              > I think you need to get a newer Vim. This was fixed around patch number 200.
              >


              FYI, the latest source is 7.3.515.

              Best regards,
              Tony.
              --
              Cigarette, n.:
              A fire at one end, a fool at the other, and a bit of tobacco in
              between.

              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php
            • Stanislav Kondratyev
              Many thanks for the tips! Now it works. I failed to found a recent Windows installer with python 3 enabled, so I had to build gvim myself (and I’m especially
              Message 6 of 6 , May 8, 2012
              • 0 Attachment
                Many thanks for the tips! Now it works.

                I failed to found a recent Windows installer with python 3 enabled, so I had to build gvim myself (and I’m especially proud of it. :)

                In case someone googles the same issue, I’d like to summarize my experience:
                http://vim.wikia.com/wiki/Build_Python-enabled_Vim_on_Windows_with_MinGW
                I edited Make_ming.mak and Makefile in their part concerning python. One can build gvim with the following command:
                mingw32-make -f Make_ming.mak gvim.exe

                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php
              Your message has been successfully submitted and would be delivered to recipients shortly.