Loading ...
Sorry, an error occurred while loading the content.

utf problems with regular vim (not macvim) on os x

Expand Messages
  • lanzz
    hello everybody, i have problems getting vim to work with unicode in OS X. it can open unicode text files fine, properly detects that they are in utf8, and
    Message 1 of 7 , Sep 28, 2007
    • 0 Attachment
      hello everybody,
      i have problems getting vim to work with unicode in OS X. it can open
      unicode text files fine, properly detects that they are in utf8, and
      displays them fine; however if i try to enter unicode (codes above
      127) via the keyboard, vim interprets the input as single-byte latin1
      input and re-encodes it from latin1 to utf8, which does not produce
      the correct results.

      vim is :set termencoding=utf8, LC_CTYPE is en_US.utf8. i've verified
      that the OS X is producing the proper utf8 sequences. utf8 works
      correctly in bash (properly displayed and text length is calculated
      correctly). the strange thing is that if i ssh from a linux box to
      the
      mac, vim works correctly with utf8. it is apparently something
      related
      to the OS X terminal, but i cannot figure out what it is, since
      typing
      "echo <some-cyrillic-text> | xxd" in shell produces exactly the same
      byte sequence on the mac as the same command on linux.

      any insights?


      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_multibyte" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • Tony Mechelynck
      ... When you do vim -N -u NONE ... what is the reply? If encoding is not set to UTF-8 at startup, and termencoding is empty, you should preserve the
      Message 2 of 7 , Sep 28, 2007
      • 0 Attachment
        lanzz wrote:
        > hello everybody,
        > i have problems getting vim to work with unicode in OS X. it can open
        > unicode text files fine, properly detects that they are in utf8, and
        > displays them fine; however if i try to enter unicode (codes above
        > 127) via the keyboard, vim interprets the input as single-byte latin1
        > input and re-encodes it from latin1 to utf8, which does not produce
        > the correct results.
        >
        > vim is :set termencoding=utf8, LC_CTYPE is en_US.utf8. i've verified
        > that the OS X is producing the proper utf8 sequences. utf8 works
        > correctly in bash (properly displayed and text length is calculated
        > correctly). the strange thing is that if i ssh from a linux box to
        > the
        > mac, vim works correctly with utf8. it is apparently something
        > related
        > to the OS X terminal, but i cannot figure out what it is, since
        > typing
        > "echo <some-cyrillic-text> | xxd" in shell produces exactly the same
        > byte sequence on the mac as the same command on linux.
        >
        > any insights?

        When you do

        vim -N -u NONE
        :verbose set enc? tenc?

        what is the reply?

        If 'encoding' is not set to UTF-8 at startup, and 'termencoding' is empty, you
        should preserve the console encoding as follows:

        if &enc !~? '^u' " nothing to do if already Unicode
        if &tenc == ''
        " avoid clobbering the keyboard encoding
        let &tenc = &enc
        endif
        set enc=utf-8

        " the following line (defaults for new files) is optional
        setglobal bomb fenc=latin1

        " set encoding detection heuristics
        set fencs=ucs-bom,utf-8,latin1
        endif



        Best regards,
        Tony.
        --
        hundred-and-one symptoms of being an internet addict:
        29. Your phone bill comes to your doorstep in a box.

        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_multibyte" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • lanzz
        On Sep 28, 4:06 pm, Tony Mechelynck ... encoding=utf-8 termencoding= --~--~---------~--~----~------------~-------~--~----~ You
        Message 3 of 7 , Sep 28, 2007
        • 0 Attachment
          On Sep 28, 4:06 pm, Tony Mechelynck <antoine.mechely...@...>
          wrote:

          > When you do
          >
          > vim -N -u NONE
          > :verbose set enc? tenc?
          >
          > what is the reply?

          encoding=utf-8
          termencoding=


          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_multibyte" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Stefán
          Hi I am probably too late to answer this thread, but I ran into the same problem. It seems to be a problem with the settings of Terminal.app . By going to the
          Message 4 of 7 , Nov 9, 2007
          • 0 Attachment
            Hi

            I am probably too late to answer this thread, but I ran into the same
            problem.

            It seems to be a problem with the settings of Terminal.app . By going
            to the menu Terminal -> Window Settings... , then the Emulation page
            there and unchecking "Escape non-ASCII characters" the problem seems
            to have disappeared. If it works, you have to click the "Use Settings
            as Default" to make the change pernanent. (Bash, after applying the
            hints at http://www.macosxhints.com/article.php?story=20060825071728278
            seems not to care about this "Escape non-ASCII characters" business.
            But that is another matter.)

            Also note that even with this solution, Terminal.app does a pretty
            lousy job of utf-8 rendering, as can be seen from trying "cat" or
            "vim" with UTF-8-demo.txt . Both mess up eg. polytonic greek pretty
            badly.

            Stefán

            On Sep 28, 5:21 am, lanzz <lanz...@...> wrote:
            > On Sep 28, 4:06 pm, Tony Mechelynck <antoine.mechely...@...>
            > wrote:
            >
            > > When you do
            >
            > > vim -N -u NONE
            > > :verbose set enc? tenc?
            >
            > > what is the reply?
            >
            > encoding=utf-8
            > termencoding=


            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_multibyte" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          • David Sewell
            ... In the Terminal.app Display preferences, are both the options for Wide glyphs unchecked? Greek and most of the other fonts display properly for me with
            Message 5 of 7 , Nov 10, 2007
            • 0 Attachment
              On Fri, 9 Nov 2007, Stefán wrote:

              > It seems to be a problem with the settings of Terminal.app . By going
              > to the menu Terminal -> Window Settings... , then the Emulation page
              > there and unchecking "Escape non-ASCII characters" the problem seems
              > to have disappeared. If it works, you have to click the "Use Settings
              > as Default" to make the change pernanent. (Bash, after applying the
              > hints at http://www.macosxhints.com/article.php?story=20060825071728278
              > seems not to care about this "Escape non-ASCII characters" business.
              > But that is another matter.)
              >
              > Also note that even with this solution, Terminal.app does a pretty
              > lousy job of utf-8 rendering, as can be seen from trying "cat" or
              > "vim" with UTF-8-demo.txt . Both mess up eg. polytonic greek pretty
              > badly.

              In the Terminal.app Display preferences, are both the options for "Wide
              glyphs" unchecked? Greek and most of the other fonts display properly
              for me with these boxes unchecked.

              --
              David Sewell, Editorial and Technical Manager
              Electronic Imprint, The University of Virginia Press
              PO Box 400318, Charlottesville, VA 22904-4318 USA
              Courier: 310 Old Ivy Way, Suite 302, Charlottesville VA 22903
              Email: dsewell@... Tel: +1 434 924 9973
              Web: http://www.ei.virginia.edu/
              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_multibyte" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            • Stefán
              Thanks Unchecking the two wide glyphs options helped a lot. Now the polytonic greek works properly. I think I finally made my Tiger terminal work as I want
              Message 6 of 7 , Nov 10, 2007
              • 0 Attachment
                Thanks

                Unchecking the two "wide glyphs" options helped a lot. Now the
                polytonic greek works properly. I think I finally made my Tiger
                terminal work as I want it. (And they just released Leopard :)

                Stefán

                On Nov 10, 6:26 am, David Sewell <dsew...@...> wrote:
                > On Fri, 9 Nov 2007, Stefán wrote:
                > > It seems to be a problem with the settings of Terminal.app . By going
                > > to the menu Terminal -> Window Settings... , then the Emulation page
                > > there and unchecking "Escape non-ASCII characters" the problem seems
                > > to have disappeared. If it works, you have to click the "Use Settings
                > > as Default" to make the change pernanent. (Bash, after applying the
                > > hints athttp://www.macosxhints.com/article.php?story=20060825071728278
                > > seems not to care about this "Escape non-ASCII characters" business.
                > > But that is another matter.)
                >
                > > Also note that even with this solution, Terminal.app does a pretty
                > > lousy job of utf-8 rendering, as can be seen from trying "cat" or
                > > "vim" with UTF-8-demo.txt . Both mess up eg. polytonic greek pretty
                > > badly.
                >
                > In the Terminal.app Display preferences, are both the options for "Wide
                > glyphs" unchecked? Greek and most of the other fonts display properly
                > for me with these boxes unchecked.
                >
                > --
                > David Sewell, Editorial and Technical Manager
                > Electronic Imprint, The University of Virginia Press
                > PO Box 400318, Charlottesville, VA 22904-4318 USA
                > Courier: 310 Old Ivy Way, Suite 302, Charlottesville VA 22903
                > Email: dsew...@... Tel: +1 434 924 9973
                > Web:http://www.ei.virginia.edu/


                --~--~---------~--~----~------------~-------~--~----~
                You received this message from the "vim_multibyte" maillist.
                For more information, visit http://www.vim.org/maillist.php
                -~----------~----~----~----~------~----~------~--~---
              • lanzz
                ... great, thanks! this fixed the problem. ... fortunately cyrillic works fine :) --~--~---------~--~----~------------~-------~--~----~ You received this
                Message 7 of 7 , Nov 15, 2007
                • 0 Attachment
                  On Nov 10, 12:14 am, Stefán <sivaldimars...@...> wrote:
                  > It seems to be a problem with the settings of Terminal.app . By going
                  > to the menu Terminal -> Window Settings... , then the Emulation page
                  > there and unchecking "Escape non-ASCII characters" the problem seems
                  > to have disappeared.

                  great, thanks! this fixed the problem.

                  > Also note that even with this solution, Terminal.app does a pretty
                  > lousy job of utf-8 rendering, as can be seen from trying "cat" or
                  > "vim" with UTF-8-demo.txt . Both mess up eg. polytonic greek pretty
                  > badly.

                  fortunately cyrillic works fine :)
                  --~--~---------~--~----~------------~-------~--~----~
                  You received this message from the "vim_multibyte" maillist.
                  For more information, visit http://www.vim.org/maillist.php
                  -~----------~----~----~----~------~----~------~--~---
                Your message has been successfully submitted and would be delivered to recipients shortly.