Loading ...
Sorry, an error occurred while loading the content.

[issue][win32]vim console version can not use the system input method under windows 8

Expand Messages
  • LiTuX
    (Seems that the one I send from gmail is blocked...) Hi all,I may meet a bug about the windows console version of vim. I ve post to vim_use
    Message 1 of 15 , Dec 8, 2013
    • 0 Attachment
      (Seems that the one I send from gmail is blocked...)
      Hi all,I may meet a bug about the windows console version of vim. I've post to vim_use (https://groups.google.com/forum/#!topic/vim_use/DfrUutlmDoM) and was suggested to post here. Details are in the link above (with screenshot), briefly, the console version can not get the correct input if IME is on.

      cream version (7.4.110) has been tested too.

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • Tony Mechelynck
      ... Any user s first post(s) to each Vim group is subject to moderation, which means that it won t appear in the mailing list until a moderator has looked at
      Message 2 of 15 , Dec 8, 2013
      • 0 Attachment
        On 08/12/13 19:20, LiTuX wrote:
        > (Seems that the one I send from gmail is blocked...)
        > Hi all,I may meet a bug about the windows console version of vim. I've post to vim_use (https://groups.google.com/forum/#!topic/vim_use/DfrUutlmDoM) and was suggested to post here. Details are in the link above (with screenshot), briefly, the console version can not get the correct input if IME is on.
        >
        > cream version (7.4.110) has been tested too.
        >

        Any user's first post(s) to each Vim group is subject to moderation,
        which means that it won't appear in the mailing list until a moderator
        has looked at it and checked that it isn't spam. The moderator can also
        (and usually will) allow posts from the same author to be posted without
        moderator filtering.

        This can take some hours for a first post (and for any messages you send
        before the first one has been approved), since moderators are few and
        not all constantly at the computer. Also, they are volunteers, and there
        is no guard schedule. Now that your first two posts to vim_dev has been
        posted, you won't have to wait as long in the future.

        This filtering process is maybe a little unpleasant, but it is the only
        solution which has been found so far in order to prevent the large
        quantity of spam sent to the Vim lists from bothering every and all
        Vim-list users. If you don't see all that spam, it is thanks to the
        moderators who stop it.


        Best regards,
        Tony.
        --
        I am, in point of fact, a particularly haughty and exclusive person, of
        pre-Adamite ancestral descent. You will understand this when I tell you
        that I can trace my ancestry back to a protoplasmal primordial atomic
        globule. Consequently, my family pride is something inconceivable. I
        can't help it. I was born sneering.
        -- Pooh-Bah, "The Mikado"

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
      • LiTuX
        Thanks for your explaination, I wonder if there is a way to merge the post here, or should I just delete the first... Update: I ve tried to use some cool stuff
        Message 3 of 15 , Dec 9, 2013
        • 0 Attachment
          Thanks for your explaination, I wonder if there is a way to merge the post here, or should I just delete the first...

          Update:
          I've tried to use some cool stuff in vim console version, which lead to the same result. You know, windows 8 comes with an onscreen keyboard, which can recognize handwrite inputs, so I tried it within vim, when the recognized result is 'inserted' into vim, it changed, the same as using MSPY.

          It might be usefull that, Microsoft use `Text Services Framework` instead of IME/IMM, and `conime` was gone, console applications were running with `conhost`. I don't know which one affects, but things may have changed, therefore using of input method in vim console version should have changed too.

          --
          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_dev" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • LiTuX
          Update: [IMPORTANT] I may now know what had happened. :) In the screenshot, I want `输入法` to be inserted into vim, but got a `嗜?`, now is why: I m using
          Message 4 of 15 , Dec 9, 2013
          • 0 Attachment
            Update: [IMPORTANT]
            I may now know what had happened. :)

            In the screenshot, I want `输入法` to be inserted into vim, but got a `嗜?`, now is why:
            I'm using code page 936 because I'm in China, hence encoding is GBK by default (and I've tested that only by :set enc=GBK can vim work normally in cmd.), hence `输入法` is actually `U+CAE4`, `U+C8EB` and `U+B7A8`, while `嗜` is `U+CAC8`, a combination of the first byte of `输入`, so, vim can get only the first byte of an input element, that is the point.

            In order to confirm, I tried to input more, for example my signature `知识共享世界大同`(something like `knowledge sharing makes the world much more better` :) ), and get `质瓜式赐`:
            知 识 共 享 世 界 大 同
            U+D6AA, U+CAB6, U+B9B2, U+CFED, U+CAC0, U+BDE7, U+B4F3, U+CDAC
            D6 CA B9 CF CA BD B4 CD
            U+D6CA U+B9CF U+CABD U+B4CD
            质 瓜 式 赐
            So it should be true.

            --
            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php

            ---
            You received this message because you are subscribed to the Google Groups "vim_dev" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
          • LiTuX
            Does anybody care about that?? -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying
            Message 5 of 15 , Dec 10, 2013
            • 0 Attachment
              Does anybody care about that??

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
            • LiTuX
              Does anybody care about that?? -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying
              Message 6 of 15 , Dec 10, 2013
              • 0 Attachment
                Does anybody care about that??

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              • mattn
                ... Try follow: 1. :echo nr2char(getchar()) 2. Type a character via your IME Can you see the same as you typed? -- -- You received this message from the
                Message 7 of 15 , Dec 10, 2013
                • 0 Attachment
                  On Tuesday, December 10, 2013 8:39:56 PM UTC+9, LiTuX wrote:
                  > Does anybody care about that??

                  Try follow:

                  1. :echo nr2char(getchar())
                  2. Type a character via your IME

                  Can you see the same as you typed?

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php

                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                • mattn
                  ... Sorry, probably, we won t help you if you won t let me know your conditions. * your settings related IME * whether did you try with vim -u NONE * your
                  Message 8 of 15 , Dec 10, 2013
                  • 0 Attachment
                    On Tuesday, December 10, 2013 8:39:56 PM UTC+9, LiTuX wrote:
                    > Does anybody care about that??

                    Sorry, probably, we won't help you if you won't let me know your conditions.

                    * your settings related IME
                    * whether did you try with vim -u NONE
                    * your windows code page
                    * whether did you try to change code page with "chcp XXX" (XXX is code page)

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/groups/opt_out.
                  • LiTuX
                    I ve already mentioned that: environment is windows 8 (from release preview to 8.1), both x86 and x64 were tested, en version with Chinese locale (which use
                    Message 9 of 15 , Dec 11, 2013
                    • 0 Attachment
                      I've already mentioned that: environment is windows 8 (from release preview to 8.1), both x86 and x64 were tested, en version with Chinese locale (which use codepage 936 instead of 437) works the same as zh-cn version.

                      For chcp, it seems that only by setting codepage to 936 can I use IME in cmd, if changed to other code page, e.g. 437, 20936, 54936, 65001, I can input English letters only. So all the test are under codepage 936, which is the default codepage for Chinese version of windows. (BTW, vim shows a mess if codepage is not 437 or 936 on my computer.) When I try to use some different IME by adding languages, I found that under codepage 936, only Chinese input method can be used in cmd (MSPY by default), Japanese don't work, and I can't change codepage to 932, so no chance to work in vim. I'll try that later.

                      It can not be caused by vimrc because I'm using the same one as in XP, and without vimrc or run vim with -u NONE lead to the same result. What's more, most vim settings related to IME seems to be used by gvim, not vim.

                      By echo nr2char(getchar()), if I use IME, the result is the same as input directly in vim.

                      --
                      --
                      You received this message from the "vim_dev" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php

                      ---
                      You received this message because you are subscribed to the Google Groups "vim_dev" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                      For more options, visit https://groups.google.com/groups/opt_out.
                    • LiTuX
                      Since msys-bash has the same issue, while msysgit s bash fixed it, I think there must be some way to fix this issue. So will anyone post a patch? -- -- You
                      Message 10 of 15 , Dec 31, 2013
                      • 0 Attachment
                        Since msys-bash has the same issue, while msysgit's bash fixed it, I think there must be some way to fix this issue. So will anyone post a patch?

                        --
                        --
                        You received this message from the "vim_dev" maillist.
                        Do not top-post! Type your reply below the text you are replying to.
                        For more information, visit http://www.vim.org/maillist.php

                        ---
                        You received this message because you are subscribed to the Google Groups "vim_dev" group.
                        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                        For more options, visit https://groups.google.com/groups/opt_out.
                      • Stephen Lee
                        ... Actually the mechanism is like that: IME(unicode) -- Console(CP936) -- vim(any codepage or utf-8) IME generates unicode characters, which tossed away by
                        Message 11 of 15 , Jan 1, 2014
                        • 0 Attachment
                          On Wednesday, January 1, 2014 6:15:18 AM UTC+8, LiTuX wrote:
                          > Since msys-bash has the same issue, while msysgit's bash fixed it, I think there must be some way to fix this issue. So will anyone post a patch?

                          Actually the mechanism is like that:
                          IME(unicode) --> Console(CP936) --> vim(any codepage or utf-8)

                          IME generates unicode characters, which tossed away by Windows Console because it can only accept codepages. Therefore you will see "different" or "corrupted" characters appear in vim.

                          This is a "live long" problem of Windows console in all versions of Windows, which Microsoft never care to fix in the past, now, or in the future.

                          So when using vim under Windows environment, please use gvim, which bypass the trouble:
                          IME(unicode) --> gvim(unicode or utf-8)

                          Cheers

                          --
                          --
                          You received this message from the "vim_dev" maillist.
                          Do not top-post! Type your reply below the text you are replying to.
                          For more information, visit http://www.vim.org/maillist.php

                          ---
                          You received this message because you are subscribed to the Google Groups "vim_dev" group.
                          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                          For more options, visit https://groups.google.com/groups/opt_out.
                        • Andre Sihera
                          ... It is not an age-old problem with Windows. The Windows Console deals in code pages (e.g. 932) by so that it can offer output that is compatible with
                          Message 12 of 15 , Jan 1, 2014
                          • 0 Attachment
                            On 01/01/14 19:59, Stephen Lee wrote:
                            > On Wednesday, January 1, 2014 6:15:18 AM UTC+8, LiTuX wrote:
                            >> Since msys-bash has the same issue, while msysgit's bash fixed it, I think there must be some way to fix this issue. So will anyone post a patch?
                            > Actually the mechanism is like that:
                            > IME(unicode) --> Console(CP936) --> vim(any codepage or utf-8)
                            >
                            > IME generates unicode characters, which tossed away by Windows Console because it can only accept codepages. Therefore you will see "different" or "corrupted" characters appear in vim.
                            >
                            > This is a "live long" problem of Windows console in all versions of Windows, which Microsoft never care to fix in the past, now, or in the future.

                            It is not an age-old problem with Windows.

                            The Windows Console deals in code pages (e.g. 932) by so that it can
                            offer output
                            that is compatible with programs that don't drive the console properly.
                            This mode
                            is called "(M)ulti(B)yte (C)haracter (S)et" mode (or MBCS) in the
                            C/Windows APIs.

                            Console applications that drive the console and I/O streams properly can
                            write
                            Unicode directly to the console (taken from IME or from another source)
                            and it
                            appears correctly.

                            My environment is Japanese and I write console programs in Unicode all
                            the time
                            and they work perfectly. Therefore it is possible to make console ViM to
                            take IME
                            input and have it appear correctly.


                            > So when using vim under Windows environment, please use gvim, which bypass the trouble:
                            > IME(unicode) --> gvim(unicode or utf-8)
                            >
                            > Cheers
                            >

                            --
                            --
                            You received this message from the "vim_dev" maillist.
                            Do not top-post! Type your reply below the text you are replying to.
                            For more information, visit http://www.vim.org/maillist.php

                            ---
                            You received this message because you are subscribed to the Google Groups "vim_dev" group.
                            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                            For more options, visit https://groups.google.com/groups/opt_out.
                          • LiTuX
                            ... Actually, the console version of vim CAN accept IME input in some earlier version of windows, e.g. Windows XP. (Of course, with a system default codepage,
                            Message 13 of 15 , Jan 1, 2014
                            • 0 Attachment
                              >
                              > This is a "live long" problem of Windows console in all versions of Windows, which Microsoft never care to fix in the past, now, or in the future.
                              >

                              Actually, the console version of vim CAN accept IME input in some earlier version of windows, e.g. Windows XP. (Of course, with a system default codepage, mine is cp936. But there are issues on input if vimrc set enc to utf8 while tenc is cp936.)

                              I found that input in vim.exe is incorrect since 2012 when I tried windows 8 Release Preview. Now from windows 8 RP to windows 8.1, x86 and x64 version all failed in my test, I thought it should be some issue about conhost (because the earlier version of windows do not use conhost, but there's conime for ime support), but because git-bash has fixed the input issue, I now believe that the input issue should be related to applications but not conhost.

                              BTW days ago, I tried to use MSYS2, which comes with a `POSIX` version of vim. That vim (with the help of msys2.dll et.al.) can accept IME input if its encoding is utf8, and leave tenc to be empty (Well, still have few issues).

                              All above, there MUST be some way to fix this issue, although it seems that a console version under windows is not so popular.

                              --
                              --
                              You received this message from the "vim_dev" maillist.
                              Do not top-post! Type your reply below the text you are replying to.
                              For more information, visit http://www.vim.org/maillist.php

                              ---
                              You received this message because you are subscribed to the Google Groups "vim_dev" group.
                              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                              For more options, visit https://groups.google.com/groups/opt_out.
                            • Stephen Lee
                              ... It is an old-age problems for windows. I know what is MBCS, and I know Windows console can output unicode without problem. You can write any unicode
                              Message 14 of 15 , Jan 1, 2014
                              • 0 Attachment
                                On Wednesday, January 1, 2014 8:16:12 PM UTC+8, Andre Sihera wrote:
                                > On 01/01/14 19:59, Stephen Lee wrote:
                                >
                                > > On Wednesday, January 1, 2014 6:15:18 AM UTC+8, LiTuX wrote:
                                >
                                > >> Since msys-bash has the same issue, while msysgit's bash fixed it, I think there must be some way to fix this issue. So will anyone post a patch?
                                >
                                > > Actually the mechanism is like that:
                                >
                                > > IME(unicode) --> Console(CP936) --> vim(any codepage or utf-8)
                                >
                                > >
                                >
                                > > IME generates unicode characters, which tossed away by Windows Console because it can only accept codepages. Therefore you will see "different" or "corrupted" characters appear in vim.
                                >
                                > >
                                >
                                > > This is a "live long" problem of Windows console in all versions of Windows, which Microsoft never care to fix in the past, now, or in the future.
                                >
                                >
                                >
                                > It is not an age-old problem with Windows.
                                >
                                >
                                >
                                > The Windows Console deals in code pages (e.g. 932) by so that it can
                                >
                                > offer output
                                >
                                > that is compatible with programs that don't drive the console properly.
                                >
                                > This mode
                                >
                                > is called "(M)ulti(B)yte (C)haracter (S)et" mode (or MBCS) in the
                                >
                                > C/Windows APIs.
                                >
                                >
                                >
                                > Console applications that drive the console and I/O streams properly can
                                >
                                > write
                                >
                                > Unicode directly to the console (taken from IME or from another source)
                                >
                                > and it
                                >
                                > appears correctly.
                                >
                                >
                                >
                                > My environment is Japanese and I write console programs in Unicode all
                                >
                                > the time
                                >
                                > and they work perfectly. Therefore it is possible to make console ViM to
                                >
                                > take IME
                                >
                                > input and have it appear correctly.
                                >
                                >
                                >
                                >
                                >
                                > > So when using vim under Windows environment, please use gvim, which bypass the trouble:
                                >
                                > > IME(unicode) --> gvim(unicode or utf-8)
                                >
                                > >
                                >
                                > > Cheers
                                >
                                > >

                                It is an old-age problems for windows.

                                I know what is MBCS, and I know Windows console can output unicode without problem. You can write any unicode directly to the console and it will display correctly.

                                What I means is it has no unicode input. Please try this by yourself:

                                Use windows console in cp936 to display japanese file with the katakana middle dot "・". This dot does not exist in cp936 but only exist in japanese (cp932).
                                Try to rename the file, or make a batch file change the filename or anything else with variables like %1,%2,etc, quoting the japanese file name.

                                You will find that you cannot because the console in cp936 does not understand the dot in cp932.

                                Please remember that NTFS uses unicode to store file names.

                                People in other forums have been discussing this issue long ago, and they also found that there is not such cp65001 (or utf-8) or cp65000 (or unicode) for the console to handle universal filename problems.

                                The only is to, like you said, making applications which drive the output part of console and let it display unicode. For the input part, Windows console can still only take in codepages, except utf-8 or unicode.

                                Windows console is still lacking a easy way to handle multilingual filenames like Linux console does.

                                --
                                --
                                You received this message from the "vim_dev" maillist.
                                Do not top-post! Type your reply below the text you are replying to.
                                For more information, visit http://www.vim.org/maillist.php

                                ---
                                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                For more options, visit https://groups.google.com/groups/opt_out.
                              • Stephen Lee
                                ... It is up to you if you persist. Personally I don t mess around with the limitation of Windows console as I have been doing that for many years. In windows
                                Message 15 of 15 , Jan 1, 2014
                                • 0 Attachment
                                  On Wednesday, January 1, 2014 10:09:05 PM UTC+8, LiTuX wrote:
                                  > >
                                  > > This is a "live long" problem of Windows console in all versions of Windows, which Microsoft never care to fix in the past, now, or in the future.
                                  > >
                                  >
                                  > Actually, the console version of vim CAN accept IME input in some earlier version of windows, e.g. Windows XP. (Of course, with a system default codepage, mine is cp936. But there are issues on input if vimrc set enc to utf8 while tenc is cp936.)
                                  >
                                  > I found that input in vim.exe is incorrect since 2012 when I tried windows 8 Release Preview. Now from windows 8 RP to windows 8.1, x86 and x64 version all failed in my test, I thought it should be some issue about conhost (because the earlier version of windows do not use conhost, but there's conime for ime support), but because git-bash has fixed the input issue, I now believe that the input issue should be related to applications but not conhost.
                                  >
                                  > BTW days ago, I tried to use MSYS2, which comes with a `POSIX` version of vim. That vim (with the help of msys2.dll et.al.) can accept IME input if its encoding is utf8, and leave tenc to be empty (Well, still have few issues).
                                  >
                                  > All above, there MUST be some way to fix this issue, although it seems that a console version under windows is not so popular.

                                  It is up to you if you persist.
                                  Personally I don't mess around with the limitation of Windows console as I have been doing that for many years.

                                  In windows I use gvim with IME plugins, and I am happy with that.

                                  Cheers

                                  --
                                  --
                                  You received this message from the "vim_dev" maillist.
                                  Do not top-post! Type your reply below the text you are replying to.
                                  For more information, visit http://www.vim.org/maillist.php

                                  ---
                                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                  For more options, visit https://groups.google.com/groups/opt_out.
                                Your message has been successfully submitted and would be delivered to recipients shortly.