Loading ...
Sorry, an error occurred while loading the content.

Re: File encoding & line ending in Vim

Expand Messages
  • Tony Mechelynck
    ... By default, Vim will detect the kind of line endings used on existing files and respect it; for new files or if you want to change the EOL for a given ...
    Message 1 of 10 , Feb 3, 2008
    • 0 Attachment
      Ciccio Bodoni wrote:
      > Hi, I'm using gvim in Italian on Windows XP SP2 (Italian + Italian
      > keyboard) with the Consolas font. At my workplace all other developers
      > have Textmate for Mac OS X. To avoid any encoding issue we decided
      > that:
      >
      > - the default file encoding has to be UTF-8
      > - the default line ending has to be LF
      >
      > (In fact these are the default settings in Textmate)
      >
      > Now I have to build a gvimrc file so that:
      >
      > - I can read, edit and save every file from Textmate (UTF-8 + LF)
      > - all new files I create are UTF-8 + LF
      >
      > I'm a newbie so any help is appreciated. Thanks in advance.
      >
      > CB

      By default, Vim will detect the kind of line endings used on existing files
      and respect it; for new files or if you want to change the EOL for a given
      files, you can specify it by ":setlocal fileformat=unix". See for more details:

      :help 'fileformats'
      :help 'fileformat'
      :help ++opt

      and what they resend to.

      For using UTF-8 with Vim, see among others
      http://vim.wikia.com/wiki/Working_with_Unicode

      Note that for UTF-8 files meant to be read on XP by Windows software, it is
      often useful to use ":setlocal bomb" so that the file starts with a BOM: for
      instance, when I was on XP SP2, I noticed that WordPad can read UTF-8 files if
      they have a BOM, but when saving "Unicode" files, what it produces is UTF-16le
      with BOM (which, BTW, Vim can also read; see the above-mentioned links).

      So here is an example of what your vimrc might look like (feel free to make
      changes). Lines starting with a double quote are comments.




      " Vim configuration file

      " Uncomment the following to force English menus and messages
      if 0
      if has("unix")
      language messages C
      else
      language messages en
      endif
      endif

      " Set various useful settings
      runtime vimrc_example.vim

      " We want to use Unicode files: can we use multibyte encodings?
      if has("multi_byte")
      " is the default locale already a Unicode one? if not, set UTF-8
      if &enc !~? '^u'
      " avoid clobbering the keyboard encoding
      if &tenc == ""
      let &tenc = &enc
      endif
      set enc=utf-8
      endif
      " define heuristics for encoding detection in existing files
      " ucs-bom (if present) must be first
      " there can be at most one 8-bit encoding (e.g. Latin1) and
      " it must come last
      set fencs=ucs-bom,utf-8,latin1
      " set encoding & BOM for newly created files
      setglobal bomb fenc=utf-8
      " create new files with Unix (LF) EOL by default
      set fileformats=unix,dos,mac
      setglobal fileformat=unix
      " warning! fileformats (plural) != fileformat (singular)
      else
      echoerr "Error= multi-byte disabled at compile time in this Vim"
      finish
      endif

      " uncomment the following line to disable filetype-specific indenting
      " filetype indent off

      " if in gvim, set the font: 5 possible formats, each version of gvim accepts
      " only one of them
      if has("gui_running")
      if has("gui_gtk2")
      set gfn=Courier\ New\ 10
      elseif has("gui_photon")
      set gfn=Courier\ New:s10
      elseif has("gui_kde")
      set gfn=Courier\ New/10/-1/5/50/0/0/0/1/0
      elseif has("x11")
      set gfn=*-courier-medium-r-normal-*-*-100-*-*-m-*-*
      else
      set gfn=Courier_New:h10:cDEFAULT
      endif
      " if gvim, start maximized
      set lines=99999 columns=99999
      endif

      " Rolodex vim: make the current window as high as possible,
      " reduce all other windows to just a status line
      set wmh=0 wh=99999

      " add additional customizations here




      Best regards,
      Tony.
      --
      When a fly lands on the ceiling, does it do a half roll or a half
      loop?


      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_use" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • Ciccio Bodoni
      Hi, thanks for your replies. I ve already created a _gvimrc file. Let s talk about some encoding related options: what are really necessary for my purposes
      Message 2 of 10 , Feb 3, 2008
      • 0 Attachment
        Hi,
        thanks for your replies. I've already created a _gvimrc file. Let's
        talk about some encoding related options: what are really necessary
        for my purposes (see my first post) and what are the values to chose?

        set encoding=utf-8

        set fileencoding=?
        set fileencodings=?
        setglobal fileencoding=utf-8

        set fileformat=?
        set fileformats=?

        let &termencoding = &encoding

        As you can see I'm a little confused among all these options :)
        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_use" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • Ciccio Bodoni
        Many thanks, especially to Ben Schmidt: my mind is really clearer now! I got things done with just three line in my _vimrc file: set encoding=utf-8 -- Every
        Message 3 of 10 , Feb 3, 2008
        • 0 Attachment
          Many thanks, especially to Ben Schmidt: my mind is really clearer now!

          I got things done with just three line in my _vimrc file:


          set encoding=utf-8

          --> Every new file I create in vim will be utf-8. Moreover vim is now
          able to read existing utf-8 files without displaying strange
          characters.


          set fileformat=unix
          --> Default line ending for new files created in vim is now LF (unix).


          set fileformats=unix,dos

          --> Please note that the default value for fileformats (ffs) is
          "dos,unix". In fact in both cases vim is able to read also files with
          dos line endings. But setting fileformats to "unix,dos" will avoid
          that existing empty files are opened as dos.


          With these simple lines my Windows machine works smoothly with my co-
          workers' Macs+Textmate (see Textmate default saving settings:
          http://macromates.com/textmate/manual/saving_files.html)

          Please tell me if my choices are right.

          CB
          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_use" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Tony Mechelynck
          ... This defines Vim s internal representation of the data. With anything else, you run the risk of not being able to represent internally some of the data
          Message 4 of 10 , Feb 3, 2008
          • 0 Attachment
            Ciccio Bodoni wrote:
            > Hi,
            > thanks for your replies. I've already created a _gvimrc file. Let's
            > talk about some encoding related options: what are really necessary
            > for my purposes (see my first post) and what are the values to chose?
            >
            > set encoding=utf-8

            This defines Vim's internal representation of the data. With anything else,
            you run the risk of not being able to represent internally some of the data
            contained in UTF-8 files.

            >
            > set fileencoding=?

            When set for an existing files, or via :setlocal, this defines how the current
            file will be represented on disk. When set "in general", or via :setglobal, it
            defines which encoding to use in the future for newly created files

            > set fileencodings=?

            this is a comma-separated list of possible encodings to be tried when opening
            an existing file. "ucs-bom" means "if the file starts with a BOM, use the
            corresponding Unicode encoding". The components are tried in left-to-right
            order, and the first one which doesn't give an error is used.

            > setglobal fileencoding=utf-8

            Future newly created files will use UTF-8 encoding

            >
            > set fileformat=?

            When set for an existing file, or via :setlocal, this defines the "style" of
            end-of-line character to be used for the current file. Thee valuyes are
            possible: unix (LF only), mac (CR only) or dos (CR+LF).

            > set fileformats=?

            This tells Vim which end-of-line styles to check for when opening an existing
            file. It is a comma-separated list but with no specific order. If empty, Vim
            won't try to determine which EOL character was used, it's the user's job to
            tell it by using ++ff=<something> when opening the file, or by setting
            'fileformat' correctly. So don't set this option to the empty string except
            temporarily, and only if you know exactly what you're doing. Many Vim users
            have been burnt before you.

            >
            > let &termencoding = &encoding

            'termencoding' defines how the keyboard (and, in console Vim, the screen)
            represents the data. The default is the empty string, which means "use the
            value of 'encoding'". This is OK as long as you don't change the 'encoding'
            value; but if you set 'encoding' to UTF-8, your keyboard won't know that, and
            will go on sending (for example) Latin1 bytes, which Vim will treat as invalid
            UTF-8 data if, like the accented letters à è ì ò ù of Italian, they are over
            0x7F. So before we change 'encoding' we must save the "old" value here so Vim
            will still "understand" what the keyboard is sending.

            >
            > As you can see I'm a little confused among all these options :)

            Well, in my previous post I gave you pointers to the Vim help and to a Vim
            wiki tip. WHY THE HELL DIDN'T YOU READ THEM AND TRY TO UNDERSTAND THEM???


            Best regards,
            Tony.
            --
            It is practically impossible to teach good programming style to
            students that have had prior exposure to BASIC: as potential
            programmers they are mentally mutilated beyond hope of
            regeneration.
            -- Dijkstra


            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_use" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          • Tony Mechelynck
            ... [...] ... Not really: even in gvim, it is relevant for the keyboard. (In console Vim it is relevant for both the keyboard and the display.) ... Since these
            Message 5 of 10 , Feb 3, 2008
            • 0 Attachment
              Ben Schmidt wrote:
              > Ciccio Bodoni wrote:
              [...]
              >> let &termencoding = &encoding
              >
              > You will need to deal with termencoding if you're using a terminal that isn't
              > UTF-8 (e.g. the Windows command window, which is a mess...). It is irrelevant if
              > the Vim GUI (gvim) is running.

              Not really: even in gvim, it is relevant for the keyboard. (In console Vim it
              is relevant for both the keyboard and the display.)

              >
              > To display the current encoding of a file, just do
              >
              > :set fenc
              >
              > To display the current line endings of a file, do
              >
              > :set ff

              Since these are string options, this will work. However, with boolean options,
              ":set diff" for instance will *set* the option rather than *display* it. To
              avoid this pitfall, I recommend to train oneself to always use a question mark
              to display the value, as in ":set fenc?" ":set ff?" ":set diff?" etc., as this
              will always display the value regardless of the option type.

              [...]
              > Hope this helps,

              Yeah, me too, and next time, Ciccio, LEGGI L'AIUTA!

              >
              > Cheers,
              >
              > Ben.

              Best regards,
              Tony.
              --
              In Greene, New York, it is illegal to eat peanuts and walk backwards on
              the sidewalks when a concert is on.


              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_use" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            • Tony Mechelynck
              ... You will be able to find out yourself. If you can t type Che sarà, sarà , Dove non c è pericolo non c è gloria , Faremo di necessità virtù etc.
              Message 6 of 10 , Feb 3, 2008
              • 0 Attachment
                Ciccio Bodoni wrote:
                > Many thanks, especially to Ben Schmidt: my mind is really clearer now!
                >
                > I got things done with just three line in my _vimrc file:
                >
                >
                > set encoding=utf-8
                >
                > --> Every new file I create in vim will be utf-8. Moreover vim is now
                > able to read existing utf-8 files without displaying strange
                > characters.
                >
                >
                > set fileformat=unix
                > --> Default line ending for new files created in vim is now LF (unix).
                >
                >
                > set fileformats=unix,dos
                >
                > --> Please note that the default value for fileformats (ffs) is
                > "dos,unix". In fact in both cases vim is able to read also files with
                > dos line endings. But setting fileformats to "unix,dos" will avoid
                > that existing empty files are opened as dos.
                >
                >
                > With these simple lines my Windows machine works smoothly with my co-
                > workers' Macs+Textmate (see Textmate default saving settings:
                > http://macromates.com/textmate/manual/saving_files.html)
                >
                > Please tell me if my choices are right.
                >
                > CB

                You will be able to find out yourself. If you can't type "Che sarà, sarà",
                "Dove non c'è pericolo non c'è gloria", "Faremo di necessità virtù" etc. with
                the correct accents, then you'll have to add a line before "set enc=utf-8" to
                avoid clobbering the 'termencoding' setting.


                Best regards,
                Tony.
                --
                GOD: That is your purpose Arthur ... the Quest for the Holy Grail ...
                "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

                --~--~---------~--~----~------------~-------~--~----~
                You received this message from the "vim_use" maillist.
                For more information, visit http://www.vim.org/maillist.php
                -~----------~----~----~----~------~----~------~--~---
              • Ben Schmidt
                ... Thanks, Tony. Correction appreciated. ... Mmm. Good call/good advice. Ben. Send instant messages to your online friends http://au.messenger.yahoo.com
                Message 7 of 10 , Feb 3, 2008
                • 0 Attachment
                  > Not really: even in gvim, it is relevant for the keyboard. (In console Vim it
                  > is relevant for both the keyboard and the display.)

                  Thanks, Tony. Correction appreciated.

                  > Since these are string options, this will work. However, with boolean options,
                  > ":set diff" for instance will *set* the option rather than *display* it. To
                  > avoid this pitfall, I recommend to train oneself to always use a question mark
                  > to display the value, as in ":set fenc?" ":set ff?" ":set diff?" etc., as this
                  > will always display the value regardless of the option type.

                  Mmm. Good call/good advice.

                  Ben.



                  Send instant messages to your online friends http://au.messenger.yahoo.com


                  --~--~---------~--~----~------------~-------~--~----~
                  You received this message from the "vim_use" maillist.
                  For more information, visit http://www.vim.org/maillist.php
                  -~----------~----~----~----~------~----~------~--~---
                • Ben Schmidt
                  ... They look to me like they will suffice. If you find you have trouble typing certain accented characters with your Italian keyboard, you will need to add
                  Message 8 of 10 , Feb 3, 2008
                  • 0 Attachment
                    > Please tell me if my choices are right.

                    They look to me like they will suffice. If you find you have trouble typing
                    certain accented characters with your Italian keyboard, you will need to add the

                    let &termencoding = &encoding

                    line to the *beginning* of your _vimrc as Tony suggested.

                    I hope you enjoy using Vim. You will find most of this becomes much easier as you
                    become more familiar with the program.

                    Ben.




                    Send instant messages to your online friends http://au.messenger.yahoo.com


                    --~--~---------~--~----~------------~-------~--~----~
                    You received this message from the "vim_use" maillist.
                    For more information, visit http://www.vim.org/maillist.php
                    -~----------~----~----~----~------~----~------~--~---
                  Your message has been successfully submitted and would be delivered to recipients shortly.