Loading ...
Sorry, an error occurred while loading the content.

Re: multibyte input

Expand Messages
  • Bram Moolenaar
    ... Hmm, I thought that Windows does support Unicode. At least on NT/2000. Isn t there a way to enter Unicode characters, e.g. in Hebrew? ... I think that
    Message 1 of 9 , Aug 9, 2000
    • 0 Attachment
      Ron Aaron wrote:

      > I'm not familiar with XIM, but the IME on Windows is for Multibyte input
      > scripts, e.g. Japanese, Korean, Chinese etc.
      >
      > Francois' problem is a little different. He wants vim to interpret his
      > keyboard input as now English, now Greek, now French. In other words, he
      > wants to switch his input encoding on the fly. XIM might do this
      > generically, but the Windows IME only does this for Far East languages.

      Hmm, I thought that Windows does support Unicode. At least on NT/2000. Isn't
      there a way to enter Unicode characters, e.g. in Hebrew?

      > Moreover, one needs to install 'support' for these languages in Windows in
      > order to remap the keyboard at Windows' level. If you don't have the Windows
      > install CD when you decide to e.g. input Spanish, you have to fake it.

      I think that Vim has to go two ways:

      1. Use the standard way to enter special characters on the system. This is
      what other programs on the same system use. And we don't have to build
      much of this into Vim.

      2. Add a good way in Vim to enter special characters, which works on any
      system. This makes it possible to enter the characters in the same way, no
      matter where you are using Vim.

      Making a compromise out of these two appears to be impossible.

      For both ways there is already some support. But now that Vim has UTF-8
      support, and the multi-byte features are improved, we probably also need to
      improve both ways. Since I don't know much about this stuff, I prefer to
      leave this to someone who knows what he's doing.

      > that's already in vim (langmap). However, as far as I can tell, 'langmap'
      > doesn't do what the documentation says. Specifically, it doesn't remap
      > INSERT mode at all, only NORMAL mode (5.7 and 6.0e on Linux, 6.0e on
      > Win32).

      Right, the idea of 'langmap' is that your keyboard is producing characters in
      your special language, thus in Insert mode nothing needs to be done. But in
      command mode you need to type keys like "x", which requires translation of
      your special language to ASCII.

      > This way, one might have a 'language' menu or command that would switch vim
      > from English to e.g. Greek input mode. Similarly, we might define an entire
      > set of 'languages' based on the standard keyboards for these.
      >
      > The problem is vim has to know that an 'alpha' is Unicode 0x03b1 but an 'a'
      > is Unicode 0x0061, and an 'alef' is Unicode 0x05d0. We should be independant
      > of the 'input method' used on the OS (for example, on DOS how does one enter
      > greek on a us keyboard?). Therefore, a remapping table is in order, to be
      > loaded for each language ("keyboard"). It should specify a translation from a
      > raw key ('a') to a translated key ('�') for a particular encoding.
      >
      > For example, a keymap for iso-8859-8/utf-8 might be something like:
      > a,0xf9,0x5e9 " a on the keyboard maps to the 'shin' character
      > b,0xf0,0x5e0 " b --> 'nun'
      > ...etc...
      >
      > Similarly for all the encodings. This means that when the user wants to (in
      > this case) enter Hebrew, he would tell vim to use a Hebrew mapping, which
      > would enter the characters as above (in utf-8, the wide characters would be
      > used, in single-byte mode, the -8 encoding would be used).

      This sounds reasonable. The question is how to handle the situation where
      there are more characters needed than you have keys on your keyboard. Perhaps
      digraphs need to be used then? As I said, I was thinking of adding a whole
      list of standard digraphs (and mnemonics).

      One problem is the handling of dead keys. Would we also want to support that?
      It's important for languages with accented characters.

      > This 'alternate' keyboard would be switched with the 'standard', English
      > keyboard via ^_ (or a user mapped key) just like it currently is for Hebrew.
      > In this manner, one can switch quickly between English and an alternate
      > language. The keyboard mapping should be able to be swtiched in on the
      > command-line as well, so searching for arbitrary text is facilitated (like
      > the current Hebrew support).

      Setting an option to the translation table to be used sounds reasonable.
      Will this replace the existing 'hkmap' and 'fkmap' options?

      More comments?

      --
      Spam seems to be something useful to novices. Later you realize that
      it's a bunch of indigestable junk that only clogs your system.
      Applies to both the food and the e-mail!

      /// Bram Moolenaar Bram@... http://www.moolenaar.net \\\
      \\\ Vim: http://www.vim.org ICCF Holland: http://iccf-holland.org ///
    • Bram Moolenaar
      ... It s indeed disappointing to see that documentation about X is hard to find. I have a CD for Mootif that contains a lot of docs, but you have to pay for
      Message 2 of 9 , Aug 9, 2000
      • 0 Attachment
        Fran�ois Thunus wrote:

        > > > goes, but I can't understand how to input. Concretely what I want to do
        > > > is to switch in the middle of a document. Say I start a sentence in
        > > > English,
        >
        > > You probably need to use the XIM (X imput method) for this. On Windows
        > > there is IME, which is similar but different.
        > I work under linux. The doc already states that I need to use XIM.
        > Now my question is: what is XIM ? how do I use it ? and alternately,
        > where do I find the doc about it ? I tried man XIM, man xim, the XF86
        > website, the X website and nowhere did I find a clue as to what I need to
        > do.

        In Vim:

        :help xim

        It's indeed disappointing to see that documentation about X is hard to find.
        I have a CD for Mootif that contains a lot of docs, but you have to pay for
        it...

        > > The text at ":help multibyte-input" should be helpful. If that is unclear
        > > or incomplete, I hope someone can improve it. I don't know enough about
        > > this myself.
        > I have read that, and multibyte.txt too. I still do not have the
        > slightest idea how to input a greek char in the middle of another text.

        Hmm, hopefully somebody can help here, if XIM doesn't do it I have no other
        idea.

        > > An alternative is to use digraphs. I'm thinking of including the RFC1345
        > > standard for digraphs. Also including the mnemonics from this table could
        > > be useful (similar to digraphs but use more than two characters).
        > the digraph method is just not practical for real typing. (that is,
        > assuming that I redefine all the letters I need). What I am looking for
        > is something like <esc>:greek
        > and lo! I can type in greek. Just like I do in Yudit: I select Greek in
        > the input menu, and I can start typing. the keyboard is remapped to the
        > closest greek equivalent, ie I get a lambda when pressing l, a phi when
        > pressing f, an alpha for a, etc. simple and fast. If I need to go back
        > to something else, I just click on input again and select the language.
        > for languages like polish, the a with a comma below is generated by a;,
        > the dashed l by a l', etc. simple enough to remember quickly.
        > What I am looking for is how to do the same in vim, if that is possible
        > at all.
        > I need to be able to change language in the middle of a text.

        Yes, I understand that. Until now Vim didn't include the translation from
        ASCII (or whatever your keyboard produces) to special languages. I was hoping
        that a system function could be used for this. See the message from/to Ron
        Aaron about an alternative.

        --
        hundred-and-one symptoms of being an internet addict:
        206. You religiously respond immediately to e-mail, while ignoring
        your growing pile of snail mail.

        /// Bram Moolenaar Bram@... http://www.moolenaar.net \\\
        \\\ Vim: http://www.vim.org ICCF Holland: http://iccf-holland.org ///
      • François Thunus
        ... YES :-) ... I work under Linux (stock RH 6.2 with some modifications for unicode, ie gazillions of fonts, patched xterm, etc). ... that would be brilliant.
        Message 3 of 9 , Aug 9, 2000
        • 0 Attachment
          Ron Aaron wrote:
          > Francois' problem is a little different. He wants vim to interpret his
          > keyboard input as now English, now Greek, now French. In other words, he
          > wants to switch his input encoding on the fly. XIM might do this generically,
          > but the Windows IME only does this for Far East languages.

          YES :-)

          > Moreover, one needs to install 'support' for these languages in Windows in
          > order to remap the keyboard at Windows' level. If you don't have the Windows
          > install CD when you decide to e.g. input Spanish, you have to fake it.

          I work under Linux (stock RH 6.2 with some modifications for unicode, ie
          gazillions of fonts, patched xterm, etc).

          > In my opinion, a much better solution is to use the inherent key remapping
          > that's already in vim (langmap). However, as far as I can tell, 'langmap'
          > doesn't do what the documentation says. Specifically, it doesn't remap INSERT
          > mode at all, only NORMAL mode (5.7 and 6.0e on Linux, 6.0e on Win32).
          >
          > This way, one might have a 'language' menu or command that would switch vim
          > from English to e.g. Greek input mode. Similarly, we might define an entire
          > set of 'languages' based on the standard keyboards for these.

          that would be brilliant.

          > The problem is vim has to know that an 'alpha' is Unicode 0x03b1 but an 'a'
          > is Unicode 0x0061, and an 'alef' is Unicode 0x05d0. We should be independant
          > of the 'input method' used on the OS (for example, on DOS how does one enter
          > greek on a us keyboard?). Therefore, a remapping table is in order, to be
          > loaded for each language ("keyboard"). It should specify a translation from a
          > raw key ('a') to a translated key ('?') for a particular encoding.
          >
          > For example, a keymap for iso-8859-8/utf-8 might be something like:
          > a,0xf9,0x5e9 " a on the keyboard maps to the 'shin' character
          > b,0xf0,0x5e0 " b --> 'nun'
          > ...etc...
          >
          > Similarly for all the encodings. This means that when the user wants to (in
          > this case) enter Hebrew, he would tell vim to use a Hebrew mapping, which
          > would enter the characters as above (in utf-8, the wide characters would be
          > used, in single-byte mode, the -8 encoding would be used).
          >
          > This gives an easy way to accomodate all systems, and encodings which are not
          > well supported. The 'rightleft' issues are a separate thing, and are already
          > adequately supported in vim I think.

          I was not even dreaming of getting that too, but if it is possible to
          reverse the direction for Hebrew and Arabic (which rae the only two I am
          interested in at the moment), that is even better than I thought.

          > This 'alternate' keyboard would be switched with the 'standard', English
          > keyboard via ^_ (or a user mapped key) just like it currently is for Hebrew.
          > In this manner, one can switch quickly between English and an alternate
          > language. The keyboard mapping should be able to be swtiched in on the
          > command-line as well, so searching for arbitrary text is facilitated (like
          > the current Hebrew support).
          > Sorry for rambling, but I think it is important to make this work
          > independantly of OS support, since it is possible, and will make vim more
          > useful to more people.

          definitely. This is exactly what I need/want, I am very glad to see that
          other people have given it a thought as well. I will see what I can do
          with this keyboard mapping option you mention. maybe that is the
          beginning of a solution for languages which are not very far from plain
          ascii (I'm thinking mostly polish, french, etc). I was not even aware
          that Hebrew was already supported by vim. I have been using unix for a
          long time, and Linux since 1.2.4, but since I was switching constantly
          between Linux, System Vr4 and Solaris, I limited myself to core vi
          functions, and did not really care about what vi I was using, since they
          were pretty much all the same to me. But now I'll be using Linux only
          and I need unicode support, which is why I jumped on vim. Knowing that
          there is also a win32 version is of utmost interest because some people
          in the company do use windows as well, so if we can standardiuze on vim,
          that would be nice.

          Thanaks for the help.

          Cheers

          François
        • François Thunus
          ... This seems to be the way to go. ... make sense ... I am not sure this does matter, in the sense that as long as the user can produce his own keymap,
          Message 4 of 9 , Aug 9, 2000
          • 0 Attachment
            Bram Moolenaar wrote:

            > 2. Add a good way in Vim to enter special characters, which works on any
            > system. This makes it possible to enter the characters in the same way, no
            > matter where you are using Vim.

            This seems to be the way to go.

            > Right, the idea of 'langmap' is that your keyboard is producing characters in
            > your special language, thus in Insert mode nothing needs to be done. But in
            > command mode you need to type keys like "x", which requires translation of
            > your special language to ASCII.

            make sense
            >
            > > This way, one might have a 'language' menu or command that would switch vim
            > > from English to e.g. Greek input mode. Similarly, we might define an entire
            > > set of 'languages' based on the standard keyboards for these.
            >
            > This sounds reasonable. The question is how to handle the situation where
            > there are more characters needed than you have keys on your keyboard. Perhaps
            I am not sure this does matter, in the sense that as long as the user
            can produce his own keymap, (s)he'll just put on it the characters (s)he
            uses most, and figure out some way to enter the rest.
            > digraphs need to be used then?
            for example, yes.

            > As I said, I was thinking of adding a whole list of standard digraphs (and mnemonics).
            I am not sure this is needed, but why not.

            > One problem is the handling of dead keys. Would we also want to support that?
            > It's important for languages with accented characters.
            you bet

            > > This 'alternate' keyboard would be switched with the 'standard', English
            > > keyboard via ^_ (or a user mapped key) just like it currently is for Hebrew.
            > > In this manner, one can switch quickly between English and an alternate
            > > language. The keyboard mapping should be able to be swtiched in on the
            > > command-line as well, so searching for arbitrary text is facilitated (like
            > > the current Hebrew support).
            >
            > Setting an option to the translation table to be used sounds reasonable.
            > Will this replace the existing 'hkmap' and 'fkmap' options?

            I am not sure how those two work, I need to examine this closer.

            > More comments?

            I am obviously more than ready to help if I can, now that my problem
            seems to be understood :-)

            Cheers

            François
          • François Thunus
            ... user-definable ? :-) set keymapperbuf=0 or set keymapperbuf=true Cheers François
            Message 5 of 9 , Aug 9, 2000
            • 0 Attachment
              Ron Aaron wrote:


              > One question -- I can see where I might have an English document and a Hebrew
              > document. It would be nice I think if the keyboard were mapped on a
              > per-buffer basis. Thoughts?

              user-definable ? :-) set keymapperbuf=0 or set keymapperbuf=true

              Cheers

              François
            • François Thunus
              ... I have done that. It does tell me that I can switch to an alternate kanji input method, but not where to find the doc to configure the one X is using to
              Message 6 of 9 , Aug 9, 2000
              • 0 Attachment
                Bram Moolenaar wrote:

                > > Now my question is: what is XIM ? how do I use it ? and alternately,
                > > where do I find the doc about it ? I tried man XIM, man xim, the XF86
                > > website, the X website and nowhere did I find a clue as to what I need to
                > > do.
                >
                > In Vim:
                >
                > :help xim
                I have done that. It does tell me that I can switch to an alternate
                kanji input method, but not where to find the doc to configure the one X
                is using to allow me to introduce greek char in the middle of an english
                text. similarly, man xim or XIM does not exist, and man X does not point
                to anything useful. Mind you, I am not bashing, just stating that the
                existing information does not allow me to do anything concrete. The
                moment I find out what to do, I'm more than willing to write a patch for
                the doc and send it to you.

                > > I need to be able to change language in the middle of a text.

                > Yes, I understand that. Until now Vim didn't include the translation from
                > ASCII (or whatever your keyboard produces) to special languages. I was hoping
                > that a system function could be used for this. See the message from/to Ron
                > Aaron about an alternative.

                Yes, Aaron's suggestion is exactly what I am looking for. Obviously, we
                have the same preoccupations :-)

                Let me know if there is anything I can do to help. While I may not be
                able to code much, I can certainly test and check/write docs

                Cheers

                François
              Your message has been successfully submitted and would be delivered to recipients shortly.