Loading ...
Sorry, an error occurred while loading the content.

Re: BOM in QuickFix window

Expand Messages
  • Tony Mechelynck
    ... Make sure your fileencodings option (plural) starts with ucs-bom . In that case, for editfiles at least (not sure about quickfix error files), Vim will
    Message 1 of 5 , Jul 8, 2009
    • 0 Attachment
      On 08/07/09 22:48, Aleksey wrote:
      >
      > Hi.
      >
      > I'm using few MONO apps (Nemerle compiler and COCO/R parser generator)
      > through compiler plugins in VIM under Linux, and the problem is that
      > they produce output starting with FEFF.
      > I had this problem about a half a year ago and being (and staying) a
      > newbie, thought that the problem was with *errorformat* but didn't
      > cope defining it correctly - not sure if it is possible.
      > I found workaround, setting LANG=en_US in compiler call command.
      >
      > I have encoding and termencoding set to 'utf-8' and locale is
      > 'en_US.utf8'.
      >
      > Is there a better way to solve this problem?
      >
      > Thanks

      Make sure your 'fileencodings' option (plural) starts with "ucs-bom". In
      that case, for editfiles at least (not sure about quickfix error files),
      Vim will recognise the Unicode codepoint U+FEFF (known as the BOM for
      Byte Order Mark though actually it's more than that) when it happens at
      the very start of a file, and set 'fileencoding' (singular) and 'bomb'
      accordingly for that file, as follows:

      Lead bytes (hex) 'fileencoding' 'bomb'
      EF BB BF utf-8 bomb
      00 00 FE FF ucs-4 bomb
      FF FE 00 00 ucs-4le bomb
      FE FF utf-16 bomb
      FF FE utf-16le bomb
      anything else not yet known nobomb

      As you can see, the BOM allows us to identify all Unicode encodings
      (endianness included, but treating UCS-2 as a particular case of UTF-16,
      which it is) assuming that no little-endian UTF-16 file will ever start
      with a NULL codepoint, which I think is a reasonable assumption. ("Not
      yet known" in the table above means "try the next entry in
      'fileencodings'".)

      If, after trying it, you find that it doesn't work for the quickfix
      errorfile, come back to report it, and in that case maybe Bram will pass
      by and add it to his TODO list. But even if he does, don't set your
      hopes too high: it's a very long list, see |todo.txt| in the Vim help
      (the file is currently 4705 lines long as of 2 July 2009, but I didn't
      count how many todo items (which should be fewer ;-) ) are in it.

      IIUC, even if 'fileencodings' doesn't work for quickfix error files,
      setting LC_MESSAGES should be enough (LANG is used as a default for any
      LC_<something> which is unset, or LC_ALL if present overrides any other
      LC_* even if present). Maybe even just using ":language messages en-US"
      or ":lang mess C" near the top of your vimrc could be enough.



      Best regards,
      Tony.
      --
      hundred-and-one symptoms of being an internet addict:
      66. You create a homepage with the impression to cure the afflicted...but
      your hidden agenda is to receive more e-mail.

      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_multibyte" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • Aleksey
      ... Thanks for your answer. fileencoding already had ucs-bom in its start, so it didnt help. I also have :language messages C, so it didn t help either. I
      Message 2 of 5 , Jul 8, 2009
      • 0 Attachment
        > Make sure your 'fileencodings' option (plural) starts with "ucs-bom".

        > IIUC, even if 'fileencodings' doesn't work for quickfix error files,
        > setting LC_MESSAGES should be enough (LANG is used as a default for any
        > LC_<something> which is unset, or LC_ALL if present overrides any other
        > LC_* even if present). Maybe even just using ":language messages en-US"
        > or ":lang mess C" near the top of your vimrc could be enough.
        >
        > Best regards,
        > Tony.
        > --
        > hundred-and-one symptoms of being an internet addict:
        > 66. You create a homepage with the impression to cure the afflicted...but
        >      your hidden agenda is to receive more e-mail.

        Thanks for your answer.

        fileencoding already had 'ucs-bom' in its start, so it didnt' help.
        I also have :language messages C, so it didn't help either.
        I didn't quite understand what you wrote about LC_ stuff, because I
        lack even basic knowledge, but does it mean my solution with setting
        LANG=en_US is fine?

        Reading docs for VIM 7.2 (:help quickfix) I found the following
        paragraph:
        When 'encoding' differs from the locale, the error messages may
        have a
        different encoding from what Vim is using. To convert the
        messages you can
        use this code:
        function QfMakeConv()
        let qflist = getqflist()
        for i in qflist
        let i.text = iconv(i.text, "cp936", "utf-8")
        endfor
        call setqflist(qflist)
        endfunction

        au QuickfixCmdPost make call QfMakeConv()

        I've tried putting this to .vimrc replacing "cp936" with ucs-bom, but
        not sure if that was right. It didn't change anything.

        I don't think this problem is worth of Bram's attention since it's not
        really wide-spread and the workaround is available.
        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_multibyte" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • Tony Mechelynck
        ... should be fileencodings with s at the end. There are two different options, with and without s, and they don t have the same meaning. Without s, it s the
        Message 3 of 5 , Jul 9, 2009
        • 0 Attachment
          On 09/07/09 08:52, Aleksey wrote:
          >
          >> Make sure your 'fileencodings' option (plural) starts with "ucs-bom".
          ------------------------------------------^^^^^^
          >
          >> IIUC, even if 'fileencodings' doesn't work for quickfix error files,
          >> setting LC_MESSAGES should be enough (LANG is used as a default for any
          >> LC_<something> which is unset, or LC_ALL if present overrides any other
          >> LC_* even if present). Maybe even just using ":language messages en-US"
          >> or ":lang mess C" near the top of your vimrc could be enough.
          >>
          >> Best regards,
          >> Tony.
          >> --
          >> hundred-and-one symptoms of being an internet addict:
          >> 66. You create a homepage with the impression to cure the afflicted...but
          >> your hidden agenda is to receive more e-mail.
          >
          > Thanks for your answer.
          >
          > fileencoding already had 'ucs-bom' in its start, so it didnt' help.

          should be 'fileencodings' with s at the end. There are two different
          options, with and without s, and they don't have the same meaning.
          Without s, it's the encoding used on disk for one file at a time. With
          s, it's the heuristics to find what to use when opening an existing file.

          > I also have :language messages C, so it didn't help either.
          > I didn't quite understand what you wrote about LC_ stuff, because I
          > lack even basic knowledge, but does it mean my solution with setting
          > LANG=en_US is fine?

          yeah, sure.

          >
          > Reading docs for VIM 7.2 (:help quickfix) I found the following
          > paragraph:
          > When 'encoding' differs from the locale, the error messages may
          > have a
          > different encoding from what Vim is using. To convert the
          > messages you can
          > use this code:
          > function QfMakeConv()
          > let qflist = getqflist()
          > for i in qflist
          > let i.text = iconv(i.text, "cp936", "utf-8")
          > endfor
          > call setqflist(qflist)
          > endfunction
          >
          > au QuickfixCmdPost make call QfMakeConv()
          >
          > I've tried putting this to .vimrc replacing "cp936" with ucs-bom, but
          > not sure if that was right. It didn't change anything.

          no, iconv doesn't know about ucs-bom.

          >
          > I don't think this problem is worth of Bram's attention since it's not
          > really wide-spread and the workaround is available.

          How can one know how widespread it is? Sounds like the age-old "Nobody
          uses it" used without proof by developers of some applications (not by
          Bram) when speaking of a feature they want to remove because they don't
          use it themselves and they don't want to support it.

          OK, so IIUC the problem seems to be: When a compiler errorfile starts
          with a BOM, Vim doesn't know how to handle it. I'm not sure if there's
          an 'errorfile' setting allowing to get over it because I've never looked
          into that option myself.


          Best regards,
          Tony.
          --
          "Benson, you are so free of the ravages of intelligence"
          -- Time Bandits

          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_multibyte" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Aleksey
          On Jul 9, 1:51 pm, Tony Mechelynck ... Sorry, mistyped it. It s fileencodings certainly, and ucs-bom is the list. ... I
          Message 4 of 5 , Jul 9, 2009
          • 0 Attachment
            On Jul 9, 1:51 pm, Tony Mechelynck <antoine.mechely...@...>
            wrote:

            > should be 'fileencodings' with s at the end. There are two different
            > options, with and without s, and they don't have the same meaning.
            > Without s, it's the encoding used on disk for one file at a time. With
            > s, it's the heuristics to find what to use when opening an existing file.

            Sorry, mistyped it. It's 'fileencodings' certainly, and 'ucs-bom' is
            the list.


            > > I don't think this problem is worth of Bram's attention since it's not
            > > really wide-spread and the workaround is available.
            >
            > How can one know how widespread it is? Sounds like the age-old "Nobody
            > uses it" used without proof by developers of some applications (not by
            > Bram) when speaking of a feature they want to remove because they don't
            > use it themselves and they don't want to support it.

            I didn't mean that. I've made such decision because I searched a lot
            for similar
            problem and found almost nothing except your recent answers to 'Match
            a BOM'
            in a file. Which deals with different issue, but it made me think
            about mine.

            I wanted to say, that since this issue has a solution and isn't
            reported by other developers,
            people know how to live with it or don't care about it. It's just
            about not wasting time on low priority stuff, IMHO.
            I'm just glad to understand what the problem is.

            >
            > OK, so IIUC the problem seems to be: When a compiler errorfile starts
            > with a BOM, Vim doesn't know how to handle it. I'm not sure if there's
            > an 'errorfile' setting allowing to get over it because I've never looked
            > into that option myself.
            >

            Yeah, you've got it right. Thank you for your time, I'll settle down
            on LANG=en_US which is just ok for me.
            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_multibyte" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          Your message has been successfully submitted and would be delivered to recipients shortly.