Loading ...
Sorry, an error occurred while loading the content.

BOM in QuickFix window

Expand Messages
  • Aleksey
    Hi. I m using few MONO apps (Nemerle compiler and COCO/R parser generator) through compiler plugins in VIM under Linux, and the problem is that they produce
    Message 1 of 5 , Jul 8, 2009
    • 0 Attachment
      Hi.

      I'm using few MONO apps (Nemerle compiler and COCO/R parser generator)
      through compiler plugins in VIM under Linux, and the problem is that
      they produce output starting with FEFF.
      I had this problem about a half a year ago and being (and staying) a
      newbie, thought that the problem was with *errorformat* but didn't
      cope defining it correctly - not sure if it is possible.
      I found workaround, setting LANG=en_US in compiler call command.

      I have encoding and termencoding set to 'utf-8' and locale is
      'en_US.utf8'.

      Is there a better way to solve this problem?

      Thanks

      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_multibyte" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • Tony Mechelynck
      ... Make sure your fileencodings option (plural) starts with ucs-bom . In that case, for editfiles at least (not sure about quickfix error files), Vim will
      Message 2 of 5 , Jul 8, 2009
      • 0 Attachment
        On 08/07/09 22:48, Aleksey wrote:
        >
        > Hi.
        >
        > I'm using few MONO apps (Nemerle compiler and COCO/R parser generator)
        > through compiler plugins in VIM under Linux, and the problem is that
        > they produce output starting with FEFF.
        > I had this problem about a half a year ago and being (and staying) a
        > newbie, thought that the problem was with *errorformat* but didn't
        > cope defining it correctly - not sure if it is possible.
        > I found workaround, setting LANG=en_US in compiler call command.
        >
        > I have encoding and termencoding set to 'utf-8' and locale is
        > 'en_US.utf8'.
        >
        > Is there a better way to solve this problem?
        >
        > Thanks

        Make sure your 'fileencodings' option (plural) starts with "ucs-bom". In
        that case, for editfiles at least (not sure about quickfix error files),
        Vim will recognise the Unicode codepoint U+FEFF (known as the BOM for
        Byte Order Mark though actually it's more than that) when it happens at
        the very start of a file, and set 'fileencoding' (singular) and 'bomb'
        accordingly for that file, as follows:

        Lead bytes (hex) 'fileencoding' 'bomb'
        EF BB BF utf-8 bomb
        00 00 FE FF ucs-4 bomb
        FF FE 00 00 ucs-4le bomb
        FE FF utf-16 bomb
        FF FE utf-16le bomb
        anything else not yet known nobomb

        As you can see, the BOM allows us to identify all Unicode encodings
        (endianness included, but treating UCS-2 as a particular case of UTF-16,
        which it is) assuming that no little-endian UTF-16 file will ever start
        with a NULL codepoint, which I think is a reasonable assumption. ("Not
        yet known" in the table above means "try the next entry in
        'fileencodings'".)

        If, after trying it, you find that it doesn't work for the quickfix
        errorfile, come back to report it, and in that case maybe Bram will pass
        by and add it to his TODO list. But even if he does, don't set your
        hopes too high: it's a very long list, see |todo.txt| in the Vim help
        (the file is currently 4705 lines long as of 2 July 2009, but I didn't
        count how many todo items (which should be fewer ;-) ) are in it.

        IIUC, even if 'fileencodings' doesn't work for quickfix error files,
        setting LC_MESSAGES should be enough (LANG is used as a default for any
        LC_<something> which is unset, or LC_ALL if present overrides any other
        LC_* even if present). Maybe even just using ":language messages en-US"
        or ":lang mess C" near the top of your vimrc could be enough.



        Best regards,
        Tony.
        --
        hundred-and-one symptoms of being an internet addict:
        66. You create a homepage with the impression to cure the afflicted...but
        your hidden agenda is to receive more e-mail.

        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_multibyte" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • Aleksey
        ... Thanks for your answer. fileencoding already had ucs-bom in its start, so it didnt help. I also have :language messages C, so it didn t help either. I
        Message 3 of 5 , Jul 8, 2009
        • 0 Attachment
          > Make sure your 'fileencodings' option (plural) starts with "ucs-bom".

          > IIUC, even if 'fileencodings' doesn't work for quickfix error files,
          > setting LC_MESSAGES should be enough (LANG is used as a default for any
          > LC_<something> which is unset, or LC_ALL if present overrides any other
          > LC_* even if present). Maybe even just using ":language messages en-US"
          > or ":lang mess C" near the top of your vimrc could be enough.
          >
          > Best regards,
          > Tony.
          > --
          > hundred-and-one symptoms of being an internet addict:
          > 66. You create a homepage with the impression to cure the afflicted...but
          >      your hidden agenda is to receive more e-mail.

          Thanks for your answer.

          fileencoding already had 'ucs-bom' in its start, so it didnt' help.
          I also have :language messages C, so it didn't help either.
          I didn't quite understand what you wrote about LC_ stuff, because I
          lack even basic knowledge, but does it mean my solution with setting
          LANG=en_US is fine?

          Reading docs for VIM 7.2 (:help quickfix) I found the following
          paragraph:
          When 'encoding' differs from the locale, the error messages may
          have a
          different encoding from what Vim is using. To convert the
          messages you can
          use this code:
          function QfMakeConv()
          let qflist = getqflist()
          for i in qflist
          let i.text = iconv(i.text, "cp936", "utf-8")
          endfor
          call setqflist(qflist)
          endfunction

          au QuickfixCmdPost make call QfMakeConv()

          I've tried putting this to .vimrc replacing "cp936" with ucs-bom, but
          not sure if that was right. It didn't change anything.

          I don't think this problem is worth of Bram's attention since it's not
          really wide-spread and the workaround is available.
          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_multibyte" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Tony Mechelynck
          ... should be fileencodings with s at the end. There are two different options, with and without s, and they don t have the same meaning. Without s, it s the
          Message 4 of 5 , Jul 9, 2009
          • 0 Attachment
            On 09/07/09 08:52, Aleksey wrote:
            >
            >> Make sure your 'fileencodings' option (plural) starts with "ucs-bom".
            ------------------------------------------^^^^^^
            >
            >> IIUC, even if 'fileencodings' doesn't work for quickfix error files,
            >> setting LC_MESSAGES should be enough (LANG is used as a default for any
            >> LC_<something> which is unset, or LC_ALL if present overrides any other
            >> LC_* even if present). Maybe even just using ":language messages en-US"
            >> or ":lang mess C" near the top of your vimrc could be enough.
            >>
            >> Best regards,
            >> Tony.
            >> --
            >> hundred-and-one symptoms of being an internet addict:
            >> 66. You create a homepage with the impression to cure the afflicted...but
            >> your hidden agenda is to receive more e-mail.
            >
            > Thanks for your answer.
            >
            > fileencoding already had 'ucs-bom' in its start, so it didnt' help.

            should be 'fileencodings' with s at the end. There are two different
            options, with and without s, and they don't have the same meaning.
            Without s, it's the encoding used on disk for one file at a time. With
            s, it's the heuristics to find what to use when opening an existing file.

            > I also have :language messages C, so it didn't help either.
            > I didn't quite understand what you wrote about LC_ stuff, because I
            > lack even basic knowledge, but does it mean my solution with setting
            > LANG=en_US is fine?

            yeah, sure.

            >
            > Reading docs for VIM 7.2 (:help quickfix) I found the following
            > paragraph:
            > When 'encoding' differs from the locale, the error messages may
            > have a
            > different encoding from what Vim is using. To convert the
            > messages you can
            > use this code:
            > function QfMakeConv()
            > let qflist = getqflist()
            > for i in qflist
            > let i.text = iconv(i.text, "cp936", "utf-8")
            > endfor
            > call setqflist(qflist)
            > endfunction
            >
            > au QuickfixCmdPost make call QfMakeConv()
            >
            > I've tried putting this to .vimrc replacing "cp936" with ucs-bom, but
            > not sure if that was right. It didn't change anything.

            no, iconv doesn't know about ucs-bom.

            >
            > I don't think this problem is worth of Bram's attention since it's not
            > really wide-spread and the workaround is available.

            How can one know how widespread it is? Sounds like the age-old "Nobody
            uses it" used without proof by developers of some applications (not by
            Bram) when speaking of a feature they want to remove because they don't
            use it themselves and they don't want to support it.

            OK, so IIUC the problem seems to be: When a compiler errorfile starts
            with a BOM, Vim doesn't know how to handle it. I'm not sure if there's
            an 'errorfile' setting allowing to get over it because I've never looked
            into that option myself.


            Best regards,
            Tony.
            --
            "Benson, you are so free of the ravages of intelligence"
            -- Time Bandits

            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_multibyte" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          • Aleksey
            On Jul 9, 1:51 pm, Tony Mechelynck ... Sorry, mistyped it. It s fileencodings certainly, and ucs-bom is the list. ... I
            Message 5 of 5 , Jul 9, 2009
            • 0 Attachment
              On Jul 9, 1:51 pm, Tony Mechelynck <antoine.mechely...@...>
              wrote:

              > should be 'fileencodings' with s at the end. There are two different
              > options, with and without s, and they don't have the same meaning.
              > Without s, it's the encoding used on disk for one file at a time. With
              > s, it's the heuristics to find what to use when opening an existing file.

              Sorry, mistyped it. It's 'fileencodings' certainly, and 'ucs-bom' is
              the list.


              > > I don't think this problem is worth of Bram's attention since it's not
              > > really wide-spread and the workaround is available.
              >
              > How can one know how widespread it is? Sounds like the age-old "Nobody
              > uses it" used without proof by developers of some applications (not by
              > Bram) when speaking of a feature they want to remove because they don't
              > use it themselves and they don't want to support it.

              I didn't mean that. I've made such decision because I searched a lot
              for similar
              problem and found almost nothing except your recent answers to 'Match
              a BOM'
              in a file. Which deals with different issue, but it made me think
              about mine.

              I wanted to say, that since this issue has a solution and isn't
              reported by other developers,
              people know how to live with it or don't care about it. It's just
              about not wasting time on low priority stuff, IMHO.
              I'm just glad to understand what the problem is.

              >
              > OK, so IIUC the problem seems to be: When a compiler errorfile starts
              > with a BOM, Vim doesn't know how to handle it. I'm not sure if there's
              > an 'errorfile' setting allowing to get over it because I've never looked
              > into that option myself.
              >

              Yeah, you've got it right. Thank you for your time, I'll settle down
              on LANG=en_US which is just ok for me.
              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_multibyte" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            Your message has been successfully submitted and would be delivered to recipients shortly.