Loading ...
Sorry, an error occurred while loading the content.

Cannot input multibyte character that has CSI

Expand Messages
  • Taro Muraoka
    Hello, Bram Moolenaar and vim-multibyte. There is one problem of multibyte character input. It is cannot input CSI (0x9B) as a part of multibyte character,
    Message 1 of 8 , Apr 13, 2000
    • 0 Attachment
      Hello, Bram Moolenaar and vim-multibyte.

      There is one problem of multibyte character input. It is cannot input
      CSI (0x9B) as a part of multibyte character, because Vim recognize CSI
      as terminal special character. So, I made a patch.

      Strategy of this patch is substitute CSI with CTRL-Q sequence. When CSI
      is inputed from keyboard (via IME), substitute it by "<CTRL-Q>x9b". And
      add a few <CTRL-Q> parsing routines where get a character as a trailbyte
      of multibyte character.

      Thanks.
      ----
      Taro Muraoka koron@...
    • Taro Muraoka
      Sorry, I miss append file. ... Problem: Cannot input multibyte character that has CSI(0x9B). Becase CSI is recognized as terminal control character.
      Message 2 of 8 , Apr 13, 2000
      • 0 Attachment
        Sorry, I miss append file.

        > Hello, Bram Moolenaar and vim-multibyte.
        >
        > There is one problem of multibyte character input. It is cannot input
        > CSI (0x9B) as a part of multibyte character, because Vim recognize CSI
        > as terminal special character. So, I made a patch.
        >
        > Strategy of this patch is substitute CSI with CTRL-Q sequence. When CSI
        > is inputed from keyboard (via IME), substitute it by "<CTRL-Q>x9b". And
        > add a few <CTRL-Q> parsing routines where get a character as a trailbyte
        > of multibyte character.
        >
        > Thanks.
        > ----
        > Taro Muraoka koron@...


        Problem: Cannot input multibyte character that has CSI(0x9B). Becase CSI
        is recognized as terminal control character.
        Solution: When CSI is inputed from keyboard, substitute it to
        "<CTRL-Q>x9b". And add some <CTRL-Q> parsing routines where get a
        character as trailbyte of multibyte.
        Files: src/edit.c src/ex_getln.c src/gui_mac.c src/gui_w32.c
        src/normal.c


        diff -c src.orig/edit.c src/edit.c
        *** src.orig/edit.c Thu Apr 13 22:07:12 2000
        --- src/edit.c Thu Apr 13 22:09:08 2000
        ***************
        *** 550,555 ****
        --- 550,557 ----
        int c2;

        c2 = safe_vgetc();
        + if (c2 == Ctrl('Q') || c2 == Ctrl('V'))
        + c2 = get_literal();
        insert_special(c, FALSE, FALSE);
        insert_special(c2, FALSE, FALSE);
        need_redraw = TRUE;
        diff -c src.orig/ex_getln.c src/ex_getln.c
        *** src.orig/ex_getln.c Thu Apr 13 22:07:14 2000
        --- src/ex_getln.c Thu Apr 13 22:09:08 2000
        ***************
        *** 1134,1139 ****
        --- 1134,1142 ----
        if (is_dbcs && IsLeadByte(c))
        {
        IObuff[1] = safe_vgetc();
        + /* enable Ctrl-V and Ctrl-Q at trail byte */
        + if (IObuff[1] == Ctrl('V') || IObuff[1] == Ctrl('Q'))
        + IObuff[1] = get_literal();
        put_on_cmdline(IObuff, 2, TRUE);
        }
        else
        diff -c src.orig/gui_mac.c src/gui_mac.c
        *** src.orig/gui_mac.c Thu Apr 13 22:07:14 2000
        --- src/gui_mac.c Thu Apr 13 22:09:08 2000
        ***************
        *** 1293,1298 ****
        --- 1293,1311 ----
        add_to_input_buf(string2, 3);
        }

        + #ifdef MULTI_BYTE
        + /*
        + * Substitute 0x9B by "<CTRL-Q>x9b". Because, normally cannot input
        + * multibyte character that has 0x9B (CSI).
        + */
        + if (string[0] == CSI)
        + {
        + const static char_u csi_str[] = { Ctrl('Q'), 'x', '9', 'b' };
        + add_to_input_buf((char_u*)csi_str, 4);
        + return;
        + }
        + #endif
        +
        add_to_input_buf(string, num);
        }

        diff -c src.orig/gui_w32.c src/gui_w32.c
        *** src.orig/gui_w32.c Thu Apr 13 22:07:14 2000
        --- src/gui_w32.c Thu Apr 13 22:09:08 2000
        ***************
        *** 826,831 ****
        --- 826,845 ----
        trash_input_buf();
        got_int = TRUE;
        }
        +
        + #ifdef MULTI_BYTE
        + /*
        + * Substitute 0x9B by "<CTRL-Q>x9b". Because, normally cannot input
        + * multibyte character that has 0x9B (CSI).
        + */
        + if (string[0] == CSI)
        + {
        + const static char_u csi_str[] = { Ctrl('Q'), 'x', '9', 'b' };
        + add_to_input_buf((char_u*)csi_str, 4);
        + return;
        + }
        + #endif
        +
        add_to_input_buf(string, 1);
        }

        diff -c src.orig/normal.c src/normal.c
        *** src.orig/normal.c Thu Apr 13 22:07:14 2000
        --- src/normal.c Thu Apr 13 22:09:08 2000
        ***************
        *** 425,431 ****
        --- 425,435 ----
        ++no_mapping;
        ++allow_keys; /* no mapping for nchar, but allow key codes */
        if (ca.cmdchar == 'g')
        + {
        ca.nchar = safe_vgetc();
        + if (ca.nchar == Ctrl('Q') || ca.nchar == Ctrl('V'))
        + ca.nchar = get_literal();
        + }
        #ifdef CURSOR_SHAPE
        if (ca.cmdchar == 'r' || (ca.cmdchar == 'g' && ca.nchar == 'r'))
        {
        ***************
        *** 437,445 ****
        --- 441,457 ----
        * extra one though.
        */
        if (ca.cmdchar != 'g')
        + {
        ca.nchar = safe_vgetc();
        + if (ca.nchar == Ctrl('Q') || ca.nchar == Ctrl('V'))
        + ca.nchar = get_literal();
        + }
        else if (ca.nchar == 'r')
        + {
        ca.extra_char = safe_vgetc();
        + if (ca.extra_char == Ctrl('Q') || ca.extra_char == Ctrl('V'))
        + ca.extra_char = get_literal();
        + }
        #ifdef CURSOR_SHAPE
        State = NORMAL_BUSY;
        #endif
        ***************
        *** 4491,4497 ****
        --- 4503,4513 ----
        int prechar = NUL; /* init for GCC */

        if (is_dbcs && IsLeadByte(cap->nchar))
        + {
        trailbyte = (char_u)safe_vgetc();
        + if (trailbyte == Ctrl('Q') || trailbyte == Ctrl('V'))
        + trailbyte = get_literal();
        + }
        #endif
        prep_redo(cap->oap->regname, cap->count1,
        NUL, 'r', had_ctrl_v, cap->nchar);
      • Bram Moolenaar
        ... Ah yes, this problem existed from the very start the GUI version was created. I wonder why you chose to solve it with the CTRL-Q sequence. An alternative
        Message 3 of 8 , Apr 13, 2000
        • 0 Attachment
          Taro Muraoka wrote:

          > Hello, Bram Moolenaar and vim-multibyte.
          >
          > There is one problem of multibyte character input. It is cannot input
          > CSI (0x9B) as a part of multibyte character, because Vim recognize CSI
          > as terminal special character. So, I made a patch.
          >
          > Strategy of this patch is substitute CSI with CTRL-Q sequence. When CSI
          > is inputed from keyboard (via IME), substitute it by "<CTRL-Q>x9b". And
          > add a few <CTRL-Q> parsing routines where get a character as a trailbyte
          > of multibyte character.

          Ah yes, this problem existed from the very start the GUI version was created.

          I wonder why you chose to solve it with the CTRL-Q sequence. An alternative
          is to add a K_CSI code. The three-byte encoding would be K_SPECIAL KS_EXTRA
          KE_CSI. It would work like K_KDEL. Is there a reason you didn't do it this
          way, or didn't you consider this alternative?

          --
          hundred-and-one symptoms of being an internet addict:
          35. Your husband tells you he's had the beard for 2 months.

          /-/-- Bram Moolenaar --- Bram@... --- http://www.moolenaar.net --\-\
          \-\-- Vim: http://www.vim.org ---- ICCF Holland: http://www.vim.org/iccf --/-/
        • Taro Muraoka
          ... Because I could not understand mechanism of three-byte encoding. So I cannot use it. CTRL-Q sequence is a way that I could implement easy. ... Taro
          Message 4 of 8 , Apr 14, 2000
          • 0 Attachment
            Bram Moolenaar:

            > I wonder why you chose to solve it with the CTRL-Q sequence. An alternative
            > is to add a K_CSI code. The three-byte encoding would be K_SPECIAL KS_EXTRA
            > KE_CSI. It would work like K_KDEL. Is there a reason you didn't do it this
            > way, or didn't you consider this alternative?

            Because I could not understand mechanism of three-byte encoding. So I
            cannot use it. CTRL-Q sequence is a way that I could implement easy.
            ----
            Taro Muraoka koron@...
          • Bram Moolenaar
            This is my alternative patch to insert CSI characters. It works like the patch from Taro Muraoka, but uses a special key byte sequence. This is more like
            Message 5 of 8 , Apr 16, 2000
            • 0 Attachment
              This is my alternative patch to insert CSI characters. It works like the
              patch from Taro Muraoka, but uses a special key byte sequence. This is more
              like it's done for other special keys.

              I added the codes <CSI> and <xCSI>, to be used in mappings. <CSI> can be used
              for terminal escape sequences, <xCSI> for a <CSI> typed in the GUI.

              Please try this out and let me know if this works or not.

              You may want to skip the parts of the patch for files that you don't have
              (gui_amiga.c, gui_mac.c, etc).

              *** ../vim-5.6.67/src/getchar.c Sat Mar 25 21:29:50 2000
              --- src/getchar.c Sun Apr 16 11:28:20 2000
              ***************
              *** 1114,1119 ****
              --- 1114,1125 ----
              continue;
              }
              #endif
              + #ifdef USE_GUI
              + /* Translate K_CSI to CSI. The special key is only used to avoid
              + * it being recognized as the start of a special key. */
              + if (c == K_CSI)
              + c = CSI;
              + #endif
              }
              #ifdef MSDOS
              /*
              *** ../vim-5.6.67/src/gui_amiga.c Sat Dec 4 19:51:22 1999
              --- src/gui_amiga.c Sun Apr 16 11:47:11 2000
              ***************
              *** 379,388 ****
              break;
              case IDCMP_VANILLAKEY:
              {
              ! char_u string[1];
              string[0] = (char_u)code;
              returnEvent = ev_KeyStroke;
              - add_to_input_buf(string,1);
              break;
              case IDCMP_RAWKEY:
              if (msg->Qualifier & IEQUALIFIER_LSHIFT)
              --- 379,397 ----
              break;
              case IDCMP_VANILLAKEY:
              {
              ! char_u string[3];
              !
              string[0] = (char_u)code;
              + if (code == CSI)
              + {
              + /* Insert CSI as K_CSI. Untested! */
              + string[1] = KS_EXTRA;
              + string[2] = KE_CSI;
              + add_to_input_buf(string, 3);
              + }
              + else
              + add_to_input_buf(string, 1);
              returnEvent = ev_KeyStroke;
              break;
              case IDCMP_RAWKEY:
              if (msg->Qualifier & IEQUALIFIER_LSHIFT)
              *** ../vim-5.6.67/src/gui_gtk_x11.c Tue Mar 28 11:58:41 2000
              --- src/gui_gtk_x11.c Sun Apr 16 11:51:34 2000
              ***************
              *** 633,638 ****
              --- 633,647 ----
              trash_input_buf();
              got_int = TRUE;
              }
              +
              + if (len == 1 && string[0] == CSI)
              + {
              + /* Turn CSI into K_CSI. */
              + string[1] = KS_EXTRA;
              + string[2] = KE_CSI;
              + len = 3;
              + }
              +
              add_to_input_buf(string, len);

              /* blank out the pointer if necessary */
              *** ../vim-5.6.67/src/gui_mac.c Thu Dec 30 10:07:36 1999
              --- src/gui_mac.c Sun Apr 16 11:39:36 2000
              ***************
              *** 1613,1624 ****
              switch (a)
              {
              case 0x1e:
              ! return ro_press('k','h', 0); /* Home */
              case 0x7f:
              ! return ro_press('k','D', 0); /* Delete */
              default:
              add_to_input_buf(code, 1);
              ! return;
              }
              case 1:
              if ((a & 0xcf) == 0xcc)
              --- 1614,1635 ----
              switch (a)
              {
              case 0x1e:
              ! ro_press('k','h', 0); /* Home */
              ! return;
              case 0x7f:
              ! ro_press('k','D', 0); /* Delete */
              ! return;
              ! case CSI:
              ! {
              ! /* Turn CSI into K_CSI. Untested! */
              ! char_u string[3] = {CSI, KS_EXTRA, KE_CSI};
              !
              ! add_to_input_buf(string, 3);
              ! return;
              ! }
              default:
              add_to_input_buf(code, 1);
              ! return;
              }
              case 1:
              if ((a & 0xcf) == 0xcc)
              *** ../vim-5.6.67/src/gui_w32.c Wed Apr 5 16:30:51 2000
              --- src/gui_w32.c Sun Apr 16 11:58:42 2000
              ***************
              *** 816,832 ****
              UINT ch,
              int cRepeat)
              {
              ! char_u string[1];

              /* TRACE("OnChar(%d, %c)\n", ch, ch); */

              string[0] = ch;
              ! if (string[0] == Ctrl('C') && !mapped_ctrl_c)
              {
              trash_input_buf();
              got_int = TRUE;
              }
              ! add_to_input_buf(string, 1);
              }

              static void
              --- 816,841 ----
              UINT ch,
              int cRepeat)
              {
              ! char_u string[3];

              /* TRACE("OnChar(%d, %c)\n", ch, ch); */

              string[0] = ch;
              ! if (ch == Ctrl('C') && !mapped_ctrl_c)
              {
              trash_input_buf();
              got_int = TRUE;
              }
              !
              ! if (ch == CSI)
              ! {
              ! /* Insert CSI as K_CSI. */
              ! string[1] = KS_EXTRA;
              ! string[2] = KE_CSI;
              ! add_to_input_buf(string, 3);
              ! }
              ! else
              ! add_to_input_buf(string, 1);
              }

              static void
              ***************
              *** 871,884 ****
              --- 880,901 ----
              string[len++] = KS_MODIFIER;
              string[len++] = modifiers;
              }
              +
              if (IS_SPECIAL(ch))
              {
              string[len++] = CSI;
              string[len++] = K_SECOND(ch);
              string[len++] = K_THIRD(ch);
              }
              + else if (ch == CSI)
              + {
              + string[len++] = CSI;
              + string[len++] = KS_EXTRA;
              + string[len++] = KE_CSI;
              + }
              else
              string[len++] = ch;
              +
              add_to_input_buf(string, len);
              }

              *** ../vim-5.6.67/src/keymap.h Thu Mar 23 17:46:04 2000
              --- src/keymap.h Sun Apr 16 11:24:45 2000
              ***************
              *** 206,212 ****
              KE_MOUSEUP, /* scroll wheel pseudo-button Up */

              KE_KINS, /* keypad Insert key */
              ! KE_KDEL /* keypad Delete key */
              };

              /*
              --- 206,214 ----
              KE_MOUSEUP, /* scroll wheel pseudo-button Up */

              KE_KINS, /* keypad Insert key */
              ! KE_KDEL, /* keypad Delete key */
              !
              ! KE_CSI /* CSI typed directly */
              };

              /*
              ***************
              *** 350,355 ****
              --- 352,359 ----

              #define K_SELECT TERMCAP2KEY(KS_SELECT, K_FILLER)
              #define K_TEAROFF TERMCAP2KEY(KS_TEAROFF, K_FILLER)
              +
              + #define K_CSI TERMCAP2KEY(KS_EXTRA, KE_CSI)

              /*
              * Symbols for pseudo keys which are translated from the real key symbols
              *** ../vim-5.6.67/src/misc2.c Thu Mar 23 17:46:04 2000
              --- src/misc2.c Sun Apr 16 12:04:07 2000
              ***************
              *** 1335,1340 ****
              --- 1335,1342 ----
              {K_BS, (char_u *)"BS"},
              {K_BS, (char_u *)"BackSpace"}, /* Alternative name */
              {ESC, (char_u *)"Esc"},
              + {CSI, (char_u *)"CSI"},
              + {K_CSI, (char_u *)"xCSI"},
              {'|', (char_u *)"Bar"},
              {'\\', (char_u *)"Bslash"},
              {K_DEL, (char_u *)"Del"},

              --
              ERROR 047: Keyboard not found. Press RETURN to continue.

              /-/-- Bram Moolenaar --- Bram@... --- http://www.moolenaar.net --\-\
              \-\-- Vim: http://www.vim.org ---- ICCF Holland: http://www.vim.org/iccf --/-/
            • Taro Muraoka
              ... I test the patch on Windows. I can input multibyte has CSI trailbyte. But I cannot input multibyte has CSI leadbyte. I typed a multibyte character with
              Message 6 of 8 , Apr 16, 2000
              • 0 Attachment
                Bram Moolenaar:

                > Please try this out and let me know if this works or not.
                >
                > You may want to skip the parts of the patch for files that you don't have
                > (gui_amiga.c, gui_mac.c, etc).

                I test the patch on Windows. I can input multibyte has CSI trailbyte.
                But I cannot input multibyte has CSI leadbyte.

                I typed a multibyte character with leadbyte CSI, and It takes one or
                more second that printed to screen. Before it printed, I typed more
                keystroke, it makes characters broken.

                And, your patch for gui_mac.c was rejected. It seems for
                gui_riscos.c doesn't it?
                ----
                Taro Muraoka koron@...
              • Taro Muraoka
                ... I checked why Bram s patch did not work. The reason is variable ch has CSI as unsined int 0xFFFFFF9B. It need cast to unsigned char (char_u) on compare.
                Message 7 of 8 , Apr 16, 2000
                • 0 Attachment
                  I wrote:

                  > I typed a multibyte character with leadbyte CSI, and It takes one or
                  > more second that printed to screen. Before it printed, I typed more
                  > keystroke, it makes characters broken.

                  I checked why Bram's patch did not work. The reason is variable ch has
                  CSI as unsined int 0xFFFFFF9B. It need cast to unsigned char (char_u)
                  on compare. Below is one answer for Windows.

                  I checked it works fine on Windows. But I did not check other plathome.
                  Thanks.
                  ----
                  Taro Muraoka koron@...


                  *** ../vim-5.6.67/src/gui_w32.c Wed Apr 5 16:30:51 2000
                  --- src/gui_w32.c Sun Apr 16 11:58:42 2000
                  ***************
                  *** 816,832 ****
                  UINT ch,
                  int cRepeat)
                  {
                  ! char_u string[1];

                  /* TRACE("OnChar(%d, %c)\n", ch, ch); */

                  string[0] = ch;
                  ! if (string[0] == Ctrl('C') && !mapped_ctrl_c)
                  {
                  trash_input_buf();
                  got_int = TRUE;
                  }
                  ! add_to_input_buf(string, 1);
                  }

                  static void
                  --- 816,841 ----
                  UINT ch,
                  int cRepeat)
                  {
                  ! char_u string[3];

                  /* TRACE("OnChar(%d, %c)\n", ch, ch); */

                  string[0] = ch;
                  ! if (ch == Ctrl('C') && !mapped_ctrl_c)
                  {
                  trash_input_buf();
                  got_int = TRUE;
                  }
                  !
                  ! if ((char_u)ch == CSI) /* cast is needed */
                  ! {
                  ! /* Insert CSI as K_CSI. */
                  ! string[1] = KS_EXTRA;
                  ! string[2] = KE_CSI;
                  ! add_to_input_buf(string, 3);
                  ! }
                  ! else
                  ! add_to_input_buf(string, 1);
                  }

                  static void
                  ***************
                  *** 871,884 ****
                  --- 880,901 ----
                  string[len++] = KS_MODIFIER;
                  string[len++] = modifiers;
                  }
                  +
                  if (IS_SPECIAL(ch))
                  {
                  string[len++] = CSI;
                  string[len++] = K_SECOND(ch);
                  string[len++] = K_THIRD(ch);
                  }
                  + else if ((char_u)ch == CSI)
                  + {
                  + string[len++] = CSI;
                  + string[len++] = KS_EXTRA;
                  + string[len++] = KE_CSI;
                  + }
                  else
                  string[len++] = ch;
                  +
                  add_to_input_buf(string, len);
                  }
                • Bram Moolenaar
                  ... And you made a fix for this in the next message. I ll include that. Thanks for testing! ... Sorry, I had to edit the patch for gui_mac.c, because I
                  Message 8 of 8 , Apr 16, 2000
                  • 0 Attachment
                    Taro Muraoka wrote:

                    > > Please try this out and let me know if this works or not.
                    > >
                    > > You may want to skip the parts of the patch for files that you don't have
                    > > (gui_amiga.c, gui_mac.c, etc).
                    >
                    > I test the patch on Windows. I can input multibyte has CSI trailbyte.
                    > But I cannot input multibyte has CSI leadbyte.
                    >
                    > I typed a multibyte character with leadbyte CSI, and It takes one or
                    > more second that printed to screen. Before it printed, I typed more
                    > keystroke, it makes characters broken.

                    And you made a fix for this in the next message. I'll include that. Thanks
                    for testing!

                    > And, your patch for gui_mac.c was rejected. It seems for
                    > gui_riscos.c doesn't it?

                    Sorry, I had to edit the patch for gui_mac.c, because I already have many more
                    changes for that file. Just ignore that part of the patch. I'll send out the
                    Mac changes later.

                    --
                    hundred-and-one symptoms of being an internet addict:
                    57. You begin to wonder how on earth your service provider is allowed to call
                    200 hours per month "unlimited."

                    /-/-- Bram Moolenaar --- Bram@... --- http://www.moolenaar.net --\-\
                    \-\-- Vim: http://www.vim.org ---- ICCF Holland: http://www.vim.org/iccf --/-/
                  Your message has been successfully submitted and would be delivered to recipients shortly.