Loading ...
Sorry, an error occurred while loading the content.

Re: utf-8 support on win32 with default code page 1250

Expand Messages
  • Glenn Maynard
    ... iconv should not be used in Windows for codepages at all. Rather, MultiByteToWideChar and WideCharToMultiByte should be used. I think code to do this for
    Message 1 of 3 , Feb 4, 2003
    • 0 Attachment
      On Tue, Feb 04, 2003 at 08:14:25PM +0100, Zbynek Winkler wrote:
      > Debuging the file saving process showed that the iconv.dll was not used
      > for the conversion at all (it was not even looked for) so it made no
      > difference if I had it installed or not. The line that decided that no
      > iconv was needed is 3118 in fileio.c - wb_flags = get_fio_flags(fenc);.
      > When changed to wb_flags = 0; iconv finally got used for (it actually
      > complained if I have not installed it) and the conversion was correct!

      iconv should not be used in Windows for codepages at all. Rather,
      MultiByteToWideChar and WideCharToMultiByte should be used. I think
      code to do this for file I/O was integrated in CVS, but there may not
      have been a release since then, so give CVS a shot.

      --
      Glenn Maynard
    • Bram Moolenaar
      ... It looks like this problem was already fixed by patch 6.1.029. The code just before the line you changed now looks like this: if (converted && (enc_utf8
      Message 2 of 3 , Feb 5, 2003
      • 0 Attachment
        Zbynek Winkler wrote:

        > A long time ago I've posted a bug on sourceforge about utf-8 support on
        > WinXP with default code page 1250 (default for czech). I've given up
        > that time because I got the feeling that I am not able to convince
        > anybody that there *is* a bug (that was also the resolution for closing
        > the bug) thinking that in the meantime someone will fix it anyway.
        >
        > Well I've tested the 6.1 and it was still there. I did a websearch about
        > utf-8 problems in vim and found
        > http://groups.yahoo.com/group/vim-multibyte/message/585 which described
        > exactly the same problem I was having. Because I really needed the utf-8
        > this time, I've decided to take the sources and give it a shot.
        >
        > Debuging the file saving process showed that the iconv.dll was not used
        > for the conversion at all (it was not even looked for) so it made no
        > difference if I had it installed or not. The line that decided that no
        > iconv was needed is 3118 in fileio.c - wb_flags = get_fio_flags(fenc);.
        > When changed to wb_flags = 0; iconv finally got used for (it actually
        > complained if I have not installed it) and the conversion was correct!
        >
        > I am not really sure what I have changed so it would probably be a good
        > idea if someone more familiar with the code reviewed it and found the
        > real cause for the wrong behavior (this probably being just a hotfix).
        > The conclusion I've made is, that the built-in process cannot handle
        > conversion from cp1250 to utf-8, so it should handle it to the iconv.dll.
        >
        > If you want me to do some more testing, please do not hesitate to
        > contact me.

        It looks like this problem was already fixed by patch 6.1.029. The code
        just before the line you changed now looks like this:

        if (converted && (enc_utf8 || STRCMP(p_enc, "latin1") == 0))

        It used to be:

        if (converted && (enc_utf8 || !has_mbyte))

        Give that change a try, I guess it fixes your problem.

        --
        hundred-and-one symptoms of being an internet addict:
        137. You decide to stay in college for an additional year or two,
        just so you can have the free Internet access.

        /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
        /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\
        \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
        \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html ///
      Your message has been successfully submitted and would be delivered to recipients shortly.