Loading ...
Sorry, an error occurred while loading the content.

Re: gvim crash when using UTF-8 locale and paste text from Windows clipboard

Expand Messages
  • Bram Moolenaar
    ... On my system I can t use the zh_CN locale, thus I can t even try to reproduce this problem. Could you at least specify the output of :version , so that we
    Message 1 of 10 , Jul 22 6:01 AM
    • 0 Attachment
      Rainux wrote (about a month ago):

      > Thanks for your help.
      >
      > I've read more help text, and done more test, I found:
      > 1. "termencoding" is never used in Win32 GUI version.
      > 2. "fileencoding" is the encoding for file, do not have relative to this case.
      > 3. When set $LANG to a UTF-8 locale, e.g. "zh-CN.UTF-8", set the
      > "encoding" to anything or let empty, then gvim will crash while paste
      > LITTLE LONG text form Windows clipboard.
      > 4. When not set $LANG, $LANG actually be "zh_CN" (the default locale I
      > use), set the "encoding" to "utf-8" or "cp936" (the code page I use),
      > gvim works fine. But, if "encoding" is "utf-8", the messages gvim
      > displayed is not correctly, because the message text gvim selected is
      > encoded in "cp936" (because the $LANG is "zh_CN", not "zh_CN.UTF-8".).
      >
      > I need set "encoding" to "utf-8", because this will let vim detect the
      > file encoding and convert it correctly. When use "cp936", the file
      > encoded in "cp950"(big5) and "ucs-2le" will not be detected and
      > converted.

      On my system I can't use the zh_CN locale, thus I can't even try to
      reproduce this problem.

      Could you at least specify the output of ":version", so that we can see
      what Vim version you are using?

      You might want to try a recent Vim, like what Tony Mechelynck makes
      available, see http://users.skynet.be/antoine.mechelynck/vim/.

      --
      A hamburger walks into a bar, and the bartender says: "I'm sorry,
      but we don't serve food here."

      /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
      /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
      \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
      \\\ Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html ///
    • Bram Moolenaar
      Rainux - ... Thus when you remove that vim.mo file it doesn t crash? That means there is a problem with a UTF-8 message handled like your codepage (was it
      Message 2 of 10 , Jul 27 2:29 PM
      • 0 Attachment
        Rainux -

        > Sorry, after more test, I found Tony's build of Vim not crash when
        > paste LITTLE LONG TEXT because it do not have a zh_CN.UTF-8 messages
        > translation module ($VIMRUNTIME/lang/zh_CN.UTF-8/LC_MESSAGES/vim.mo).
        >
        > I copied this file and "libintl.dll" from official version to Tony's
        > build, then do the paste test, it crashes.
        >
        > I removed this file from the official version, then the official
        > version don't crash in the paste test, like Tony's.
        >
        > Hope this little information can help you to find and fix this
        > "bug"(or not Vim's bug?), Bram. I'm sorry I've observed you're the
        > author of Vim just now. ;)

        Thus when you remove that vim.mo file it doesn't crash? That means
        there is a problem with a UTF-8 message handled like your codepage (was
        it cp936?). Someone reported before that the sprintf() in the MS
        library causes this problem. I changed several calls to vim_snprintf()
        and checked the messages for using a "%" in the second byte, what else
        would need to be done?

        If you can, please try to reproduce the crash in a debugger, so that we
        can see where it happens. Tony's site gives hints how to compile Vim
        with free tools.

        - Bram

        --
        To keep milk from turning sour: Keep it in the cow.

        /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
        /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
        \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
        \\\ Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html ///
      • Bram Moolenaar
        Rainux - ... It seems the stack was messed up. ... It s possibly called somewhere inside a library function. ... Try setting breakpoints to further pinpoint
        Message 3 of 10 , Jul 28 2:43 PM
        • 0 Attachment
          Rainux -

          > Hi, I've used Cygwin's gcc to compiled a gvimd.exe, and tried to
          > reproduce the crash in gdb, here is the conversation with gdb.
          >
          > M:\vim\vim63>gdb gvimd.exe -q
          > (gdb) run
          > Starting program: /cygdrive/m/vim/vim63/gvimd.exe
          >
          > (Here I switched to gVim and paste LITTLE LONG TEXT from Windows
          > clipboard, then gdb caught the exception.)
          >
          > Program received signal SIGSEGV, Segmentation fault.
          > 0x77c12a16 in wscanf () from /cygdrive/c/WINDOWS/system32/msvcrt.dll
          > (gdb) info stack
          > #0 0x77c12a16 in wscanf () from /cygdrive/c/WINDOWS/system32/msvcrt.dll
          > #1 0x610e8987 in ?? ()
          > #2 0x0054f020 in p_report ()
          > #3 0x0022fde0 in ?? ()

          It seems the stack was messed up.

          > It seems the crash it not Vim's bug? And, I've viewed gvimd.exe and
          > libintl.dll 's import functions from msvcrt.dll, I havn't founded the
          > function wscanf(), why?

          It's possibly called somewhere inside a library function.

          > Do you need more infomation and more action? I'm just a newbie to
          > machine-level-debugging.

          Try setting breakpoints to further pinpoint where the crash occurs.
          Perhaps clip_mch_request_selection() is a good one in this case:

          break clip_mch_request_selection

          - Bram

          --
          Bad programs can be written in any language.

          /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
          /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
          \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
          \\\ Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html ///
        • Bram Moolenaar
          ... I m glad you were able pin it down this far. That helps a lot. ... I first thought msg_buf could be too small, but you have the same problem with
          Message 4 of 10 , Jul 29 12:15 PM
          • 0 Attachment
            Rainux wrote:

            > Thanks for your hint, use this breakpoint I did a source-level trace
            > and found a strange thing.

            I'm glad you were able pin it down this far. That helps a lot.

            > The sources I used is full Unix sources, with patch 1-85 applied,
            > compiled by gcc in Cygwin (follow Tony's Compiling HowTo).
            >
            > The correct description of the "bug" is follow:
            > When yank or put lines to/from Vim's register(It's independent of
            > Windows clipboard.), if lines more than 2(or bigger?), Vim will
            > display a message such as "8 lines yanked" or "10 more lines". When
            > using zh_CN.UTF-8 locale, Vim will try to display the localized
            > string of the message, the localized yank message will be OK, but
            > try to display the localized put message will cause Vim crash.
            >
            > The yank message is print by ops.c:2871, and the put message is print by
            > misc1.c:3028.
            >
            > ops.c:2871
            > smsg((char_u *)_("%ld lines yanked"), yanklines);
            > // ^ This line display the localized yank message.
            >
            > misc1.c:3028
            > sprintf((char *)msg_buf, _("%ld more lines"), pn);
            > // ^ This line try to format the localized put message but
            > // crash in wscanf().
            > else
            > sprintf((char *)msg_buf, _("%ld fewer lines"), pn);
            >
            > The conversation with gdb:

            I first thought "msg_buf" could be too small, but you have the same
            problem with smsg(), thus that's not the case.

            There must be a problem in gettext(), which is what _() translates into.
            I'm afraid I can't do anything for that. This is the libintl.dll
            library. You could try getting another version of libintl.dll. Are you
            using the one in the Vim distribution?

            You could split up the code to make sure the crash is inside
            gettext(). Change:

            sprintf((char *)msg_buf, _("%ld more lines"), pn);

            To:
            {
            char *ts = _("%ld more lines");
            sprintf((char *)msg_buf, ts, pn);
            }

            If it now crashes in sprintf() then it's a different problem. But from
            your description it should crash in _().

            > Program received signal SIGSEGV, Segmentation fault.
            > 0x77c12a16 in wscanf () from /cygdrive/c/WINDOWS/system32/msvcrt.dll

            I guess it might have something to do with this msvcrt.dll being
            incompatible with the libintl.dll. Try using the libintl.dll that comes
            with cygwin.

            --
            BLACK KNIGHT: None shall pass.
            ARTHUR: I have no quarrel with you, brave Sir knight, but I must cross
            this bridge.
            BLACK KNIGHT: Then you shall die.
            "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

            /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
            /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
            \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
            \\\ Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html ///
          • Bram Moolenaar
            ... The problem may be that there is a % character in the second byte of one of the Chinese double-byte characters. sprintf() then may see the wrong
            Message 5 of 10 , Sep 22, 2005
            • 0 Attachment
              Rainux wrote:

              > I'm sorry for some personal reason I've left some days.
              >
              > > {
              > > char *ts =3D _("%ld more lines");
              > > sprintf((char *)msg_buf, ts, pn);
              > > }
              > >
              > > If it now crashes in sprintf() then it's a different problem. But from
              > > your description it should crash in _().
              >
              > I did this, and it crash in sprintf(), seems the problem is in msvcrt.dll.
              >
              > I use Borland C++ 5.5.1 for Win32 compile Vim just now(previously I
              > use Cygwin's gcc), and found the new gvim.exe's import table don't
              > have msvcrt.dll. And, this build of gvim never crashes in zh_CN.UTF-8.
              > So I think the problem should in msvcrt.dll(Bram use MSVC compile Vim,
              > so the "official build" rely on msvcrt.dll).

              The problem may be that there is a % character in the second byte of one
              of the Chinese double-byte characters. sprintf() then may see the wrong
              arguments.

              Another possibility is that msg_buf is too small. I made it longer a
              short while ago. Currently it's 480 bytes. In Vim 6.3 the value was
              80. Patch 6.3.072 changed it to 240. If you are still using 80 then
              this is probably the cause of the trouble.

              --
              "Hegel was right when he said that we learn from history that man can
              never learn anything from history." (George Bernard Shaw)

              /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
              /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
              \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
              \\\ Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html ///
            • Bram Moolenaar
              Rainux - ... I plan to switch to the free compiler that MS distributes. But I need to make sure there are no legal problems and find a way to debug the
              Message 6 of 10 , Nov 23, 2005
              • 0 Attachment
                Rainux -

                > Hi Bram, recently I use MSVC 6.0 to compile Vim, and found Vim 6.4
                > will not crash when try to display some UTF-8 encoded Chinese message.
                > Both GCC 3.4.4 in Cygwin and MSVC 5.0 (you use it compile the
                > "official build", don't you?) compiled Vim 6.4 will crash on this
                > case.
                >
                > But Vim 7.0aa already fixed this "bug" (I've tested above version
                > 7.0149), use GCC to compile is OK. And, Vim 7 can correctly hand the
                > Chinese file name on zh_CN.UTF-8 locale now. (Vim 6.4 can't)
                >
                > So, I want to suggest you use the recent version of MSVC to compile
                > the "official build" if possible, thanks.

                I plan to switch to the free compiler that MS distributes. But I need
                to make sure there are no legal problems and find a way to debug the
                resulting program.

                - Bram

                --
                "Hit any key to continue" it said, but nothing happened after F sharp.

                /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                \\\ download, build and distribute -- http://www.A-A-P.org ///
                \\\ help me help AIDS victims -- http://www.ICCF.nl ///
              • Rainux
                The free Visual C++ Toolkit 2003 do not contain platform SDK, therefor, you should get the newer .lib and .h files another way ;) I ve used Visual C++ 2003
                Message 7 of 10 , Nov 23, 2005
                • 0 Attachment
                  The free Visual C++ Toolkit 2003 do not contain platform SDK,
                  therefor, you should get the newer .lib and .h files another way ;)

                  I've used Visual C++ 2003 (the compiler and the one in the Visual C++
                  Toolkit 2003 is just the same) to compile Vim for some days, and it
                  seems work fine.

                  Gdb can be used as debugger in Win32 too.

                  2005/11/23, Bram Moolenaar <Bram@...>:
                  >
                  > I plan to switch to the free compiler that MS distributes. But I need
                  > to make sure there are no legal problems and find a way to debug the
                  > resulting program.
                  >
                  > - Bram
                  >
                  > --
                  > "Hit any key to continue" it said, but nothing happened after F sharp.
                  >
                  > /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                  > /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                  > \\\ download, build and distribute -- http://www.A-A-P.org ///
                  > \\\ help me help AIDS victims -- http://www.ICCF.nl ///
                  >


                  --
                  Goddess light my path!
                Your message has been successfully submitted and would be delivered to recipients shortly.