Loading ...
Sorry, an error occurred while loading the content.
 

Re: win2k IME, multibyte/width characters, etc. (patch)

Expand Messages
  • Glenn F. Maynard
    I put together a WM_IME_COMPOSITION handler; it works for Japanese input. Some quirks. You must use the *W wide versions of ImmGetCompositionString, or it ll
    Message 1 of 54 , Sep 22, 2001
      I put together a WM_IME_COMPOSITION handler; it works for Japanese input.

      Some quirks. You must use the *W wide versions of ImmGetCompositionString,
      or it'll get confused and drop everything to "?"s. This isn't that big of
      a problem, since this IME is Win2K and up only, which always have Unicode.

      Pasting is dumping raw UTF-16 into the file; hiragana for "a" (0x82CC)
      shows up as <82><CC>, rather than UTF-8. I don't know whose job it is
      to convert this.

      GetResultStr now converts from UTF-16 to UTF-8, for IME input, and no
      longer displays the string; just keeps it around for inserting.

      Nuked GetCompositionStr; it was unused, and only differed from GetResultStr
      by one constant. Added a param to GetResultStr, in case that value is
      needed later.

      The Korean input stuff has similar unreferenced stuff. It does stuff
      when bInComposition is true, but it's never set. It used lpCompStr
      where it was very unobvious where it was being set, if at all. I
      doubt it did anything, but since I deglobalized lpCompStr, I wasn't
      sure what to do with it. I munged it a bit--it "looks right", but,
      again, it doesn't seem to be used and I have no way of testing it.

      Anyone out there care to volunteer a way to test these odd keyboards
      without having one? The JP IME2000 is easy; it's a conversion mode
      for inputting JP on an English keyboard. As far as I can tell, there's
      no equivalent for Chinese or Korean.

      The arbitrary size constraints on IME input are gone.

      In _OnImeNotify, I don't know what the State & INSERT check is for
      (doing stuff in edit mode?); I wasn't able to get it to do anything.

      _OnImeNotify -> IMN_SETOPENSTATUS now returns 0; we need to call the
      DefWindowProc for it or it won't be enabled completely.

      I thought up some clever idea of maintaining the input state in edit mode,
      but always returning to English in command mode. Then I noticed you
      already did this. Damn.

      Here's the current patch. Still havn't touched the terminal stuff,
      pasting stuff, or font stuff. Sorry if it doesn't apply cleanly; I
      had to wrestle with patch to get a clean diff off of the DOS-ized
      source.

      --
      Glenn Maynard
    • Muraoka Taro
      Sorry, I forgot to attach that two files. ... diff -cr src.orig/mbyte.c src/mbyte.c ... *************** ... if (enc_utf8 && !option_was_set((char_u *) fencs ))
      Message 54 of 54 , Oct 25, 2001
        Sorry, I forgot to attach that two files.

        > I had found two problems about Glenn's latest patch for IME.
        >
        > 1. In mbyte.c FEAT_WINDOWS is used for instead of WIN32.
        > It cause touble when compiling on non-windows platform.
        > Check attached ime_fix1.diff.
        > 2. I got report of strange behavior from an user of old IME (it is called SKK).
        > I had checked this problem.
        > SKK send WM_IME_COMPOSITION message when 'k' typed
        > without any composition string.
        > It cause unexpected two white space insertion.
        > I had fixed this.
        > Please check _OnImeComposition() and GetResultStr() in ime_fix2.diff.
        >
        > Regards.
        > ----
        > Muraoka Taro <koron@...>
        >

        ----ime_fix1.diff
        diff -cr src.orig/mbyte.c src/mbyte.c
        *** src.orig/mbyte.c Sun Oct 21 20:30:38 2001
        --- src/mbyte.c Sun Oct 21 20:30:33 2001
        ***************
        *** 572,577 ****
        --- 572,591 ----
        if (enc_utf8 && !option_was_set((char_u *)"fencs"))
        set_string_option_direct((char_u *)"fencs", -1,
        (char_u *)"ucs-bom,utf-8,latin1", OPT_FREE);
        + #ifdef FEAT_MBYTE_IME
        + # ifdef USE_ICONV
        + ime_conv.vc_fd = (iconv_t)-1;
        + # endif
        + convert_setup(&ime_conv, "ucs-2", p_enc);
        + #endif
        +
        + #ifdef FEAT_MBYTE_IME
        + # ifdef USE_ICONV
        + ime_conv_cp.vc_fd = (iconv_t)-1;
        + # endif
        + ime_conv_cp.vc_type = CONV_CODEPAGE;
        + ime_conv_cp.vc_factor = 2; /* we don't really know anything about the codepage */
        + #endif

        #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
        /* GNU gettext 0.10.37 supports this feature: set the codeset used for
        ***************
        *** 3539,3566 ****
        add_to_input_buf(str, 3);
        }

        - /*
        - * Add "str[len]" to the input buffer while escaping CSI bytes.
        - */
        - static void
        - add_to_input_buf_csi(char_u *str, int len)
        - {
        - int i;
        - char_u buf[2];
        -
        - for (i = 0; i < len; ++i)
        - {
        - add_to_input_buf(str + i, 1);
        - if (str[i] == CSI)
        - {
        - /* Turn CSI into K_CSI. */
        - buf[0] = KS_EXTRA;
        - buf[1] = (int)KE_CSI;
        - add_to_input_buf(buf, 2);
        - }
        - }
        - }
        -
        static GSList *key_press_event_queue = NULL;
        static int preedit_buf_len = 0;
        static gboolean processing_queued_event = FALSE;
        --- 3553,3558 ----
        ***************
        *** 3897,3902 ****
        --- 3889,3908 ----
        /* Internal utf-8 -> latin1 conversion. */
        vcp->vc_type = CONV_TO_LATIN1;
        }
        + #ifdef WIN32
        + /* Win32-specific UTF-16 -> DBCS conversion, for the IME,
        + * so we don't need iconv ... */
        + else if((from_prop & ENC_UNICODE) && (from_prop & ENC_2BYTE) && (to_prop & ENC_DBCS))
        + {
        + vcp->vc_type = CONV_DBCS;
        + vcp->vc_factor = 2; /* up to twice as long */
        + vcp->dbcs = atoi(to + 2);
        + } else if((from_prop & ENC_UNICODE) && (from_prop & ENC_2BYTE) && (to_prop & ENC_UNICODE)) {
        + vcp->vc_type = CONV_DBCS;
        + vcp->vc_factor = 2; /* up to twice as long */
        + vcp->dbcs = CP_UTF8;
        + }
        + #endif
        # ifdef USE_ICONV
        else
        {
        ***************
        *** 4023,4029 ****
        --- 4029,4077 ----
        retval = iconv_string(vcp->vc_fd, ptr, len);
        if (retval != NULL && lenp != NULL)
        *lenp = (int)STRLEN(retval);
        + break;
        # endif
        + # ifdef WIN32
        + case CONV_DBCS: /* UTF-16 -> dbcs or UTF8 */
        + {
        + int retlen;
        + if (!lenp)
        + len /= sizeof(unsigned short);
        +
        + retlen = WideCharToMultiByte(vcp->dbcs, 0,
        + (const unsigned short *)ptr, len, 0, 0, 0, 0);
        +
        + retval = alloc(retlen + 1);
        + if (retval == NULL)
        + break;
        +
        + WideCharToMultiByte(vcp->dbcs, 0,
        + (const unsigned short *) ptr, len, retval, retlen, 0, 0);
        +
        + retval[retlen] = NUL;
        + if (lenp != NULL)
        + *lenp = retlen;
        + break;
        + }
        + case CONV_CODEPAGE: /* current codepage -> ucs-2 */
        + {
        + int retlen;
        + retlen = MultiByteToWideChar(GetACP(), 0, ptr, len,
        + 0, 0);
        +
        + retval = alloc(sizeof(unsigned short) * retlen);
        + if (retval == NULL)
        + break;
        +
        + MultiByteToWideChar(GetACP(), 0, ptr, len,
        + (unsigned short *) retval, retlen);
        +
        + if (lenp != NULL)
        + *lenp = retlen * sizeof(unsigned short); /* number of shorts -> buffer size */
        + }
        +
        + # endif
        +
        }

        return retval;


        ----ime_fix2.diff
        diff -cr src.orig/gui_w32.c src/gui_w32.c
        *** src.orig/gui_w32.c Wed Sep 19 21:43:49 2001
        --- src/gui_w32.c Tue Oct 2 17:31:11 2001
        ***************
        *** 210,215 ****
        --- 210,219 ----
        static int get_toolbar_bitmap(vimmenu_T *menu);
        #endif

        + #ifdef FEAT_MBYTE_IME
        + static LRESULT _OnImeComposition(HWND hwnd, WPARAM dbcs, LPARAM param);
        + static char_u *GetResultStr(HWND hwnd, int GCS);
        + #endif
        #if defined(FEAT_MBYTE_IME) && defined(DYNAMIC_IME)
        # ifdef NOIME
        typedef struct tagCOMPOSITIONFORM {
        ***************
        *** 221,227 ****
        # endif

        HINSTANCE hLibImm = NULL;
        ! LONG (WINAPI *pImmGetCompositionString)(HIMC, DWORD, LPVOID, DWORD);
        HIMC (WINAPI *pImmGetContext)(HWND);
        BOOL (WINAPI *pImmReleaseContext)(HWND, HIMC);
        BOOL (WINAPI *pImmGetOpenStatus)(HIMC);
        --- 225,232 ----
        # endif

        HINSTANCE hLibImm = NULL;
        ! LONG (WINAPI *pImmGetCompositionStringA)(HIMC, DWORD, LPVOID, DWORD);
        ! LONG (WINAPI *pImmGetCompositionStringW)(HIMC, DWORD, LPVOID, DWORD);
        HIMC (WINAPI *pImmGetContext)(HWND);
        BOOL (WINAPI *pImmReleaseContext)(HWND, HIMC);
        BOOL (WINAPI *pImmGetOpenStatus)(HIMC);
        ***************
        *** 232,238 ****
        BOOL (WINAPI *pImmGetConversionStatus)(HIMC, LPDWORD, LPDWORD);
        static void dyn_imm_load(void);
        #else
        ! # define pImmGetCompositionString ImmGetCompositionStringA
        # define pImmGetContext ImmGetContext
        # define pImmReleaseContext ImmReleaseContext
        # define pImmGetOpenStatus ImmGetOpenStatus
        --- 237,244 ----
        BOOL (WINAPI *pImmGetConversionStatus)(HIMC, LPDWORD, LPDWORD);
        static void dyn_imm_load(void);
        #else
        ! # define pImmGetCompositionStringA ImmGetCompositionStringA
        ! # define pImmGetCompositionStringW ImmGetCompositionStringW
        # define pImmGetContext ImmGetContext
        # define pImmReleaseContext ImmReleaseContext
        # define pImmGetOpenStatus ImmGetOpenStatus
        ***************
        *** 790,795 ****
        --- 841,852 ----
        case WM_IME_NOTIFY:
        if (!_OnImeNotify(hwnd, (DWORD)wParam, (DWORD)lParam))
        return DefWindowProc(hwnd, uMsg, wParam, lParam);
        + break;
        +
        + case WM_IME_COMPOSITION:
        + if (!_OnImeComposition(hwnd, wParam, lParam))
        + return DefWindowProc(hwnd, uMsg, wParam, lParam);
        + break;
        #endif

        default:
        ***************
        *** 1292,1298 ****
        #include <ime.h>
        #include <imm.h>

        - static char lpCompStr[100]; // Pointer to composition str.
        static BOOL bInComposition=FALSE;

        /*
        --- 1352,1357 ----
        ***************
        *** 1356,1453 ****
        }
        }
        gui_update_cursor(TRUE, FALSE);
        ! lResult = 1;
        break;
        }
        pImmReleaseContext(hWnd, hImc);
        return lResult;
        }

        ! /* get composition string from WIN_IME */
        ! static void
        ! GetCompositionStr(HWND hwnd, LPARAM CompFlag)
        {
        ! DWORD dwBufLen; // Stogare for len. of composition str
        ! HIMC hIMC; // Input context handle.
        !
        ! // If fail to get input context handle then do nothing.
        ! // Applications should call ImmGetContext API to get
        ! // input context handle.

        ! if (!pImmGetContext || !(hIMC = pImmGetContext(hwnd)))
        ! return;

        ! // Determines how much memory space to store the composition string.
        ! // Applications should call ImmGetCompositionString with
        ! // GCS_COMPSTR flag on, buffer length zero, to get the bullfer
        ! // length.

        ! if ((dwBufLen = pImmGetCompositionString(hIMC, GCS_COMPSTR,
        ! (void FAR*)NULL, 0l)) < 0)
        ! goto exit2;

        ! if (dwBufLen > 99)
        ! goto exit2;

        ! // Reads in the composition string.
        ! if ( dwBufLen != 0 )
        {
        ! pImmGetCompositionString(hIMC, GCS_COMPSTR, lpCompStr, dwBufLen);
        ! lpCompStr[dwBufLen] = 0;
        }
        else
        {
        ! strcpy(lpCompStr, " ");
        ! dwBufLen = 2;
        ! }
        !
        ! // Display new composition chars.
        ! DisplayCompStringOpaque(lpCompStr, dwBufLen);


        ! exit2:
        ! pImmReleaseContext(hwnd, hIMC);
        }

        !
        ! // void GetResultStr()
        ! //
        ! // This handles WM_IME_COMPOSITION with GCS_RESULTSTR flag on.
        ! //
        ! // get complete composition string
        !
        ! static void
        ! GetResultStr(HWND hwnd)
        {
        ! DWORD dwBufLen; // Storage for length of result str.
        ! HIMC hIMC; // Input context handle.

        - // If fail to get input context handle then do nothing.
        if (!pImmGetContext || !(hIMC = pImmGetContext(hwnd)))
        ! return;
        !
        ! // Determines how much memory space to store the result string.
        ! // Applications should call ImmGetCompositionString with
        ! // GCS_RESULTSTR flag on, buffer length zero, to get the bullfer
        ! // length.
        ! if ((dwBufLen = pImmGetCompositionString(hIMC, GCS_RESULTSTR,
        ! (void FAR *)NULL, (DWORD) 0)) <= 0)
        ! goto exit2;
        !
        ! if (dwBufLen > 99)
        ! goto exit2;
        !
        ! // Reads in the result string.
        ! pImmGetCompositionString(hIMC, GCS_RESULTSTR, lpCompStr, dwBufLen);

        ! // Displays the result string.
        ! DisplayCompStringOpaque(lpCompStr, dwBufLen);

        ! exit2:
        pImmReleaseContext(hwnd, hIMC);
        }

        ! static char *
        ImeGetTempComposition(void)
        {
        if (bInComposition == TRUE)
        --- 1415,1523 ----
        }
        }
        gui_update_cursor(TRUE, FALSE);
        ! lResult = 0;
        break;
        }
        pImmReleaseContext(hWnd, hImc);
        return lResult;
        }

        ! static LRESULT
        ! _OnImeComposition(HWND hwnd, WPARAM dbcs, LPARAM param)
        {
        ! char_u *ret;
        ! if ((param & GCS_RESULTSTR) == 0) /* Composition unfinished. */
        ! return 0;

        ! if (ret = GetResultStr(hwnd, GCS_RESULTSTR))
        ! {
        ! add_to_input_buf_csi(ret, strlen(ret));
        ! vim_free(ret);
        ! return 1;
        ! }
        ! else
        ! return 0;
        ! }

        ! /* get the currnet composition string, in UCS-2; len is the number of
        ! * Unicode characters */
        ! unsigned short *
        ! GetCompositionString_inUCS2(HIMC hIMC, DWORD GCS, int *len)
        ! {
        ! LONG ret;
        ! unsigned short *wbuf = NULL;

        ! if (!pImmGetContext)
        ! return NULL; /* no imm32.dll */

        ! /* Try Unicode; this'll always work on NT regardless of codepage. */
        ! ret = pImmGetCompositionStringW(hIMC, GCS, NULL, 0);
        ! if (ret == 0)
        ! return NULL; /* empty */

        ! if (ret > 0)
        {
        ! wbuf = (unsigned short *) alloc(ret * sizeof(unsigned short));
        ! if(!wbuf) return NULL;
        !
        ! pImmGetCompositionStringW(hIMC, GCS, wbuf, ret);
        ! *len = ret / sizeof(unsigned short); /* char -> wchar */
        ! return wbuf;
        }
        +
        + /* ret < 0; we got an error, so try the ANSI version. This'll work
        + * on 9x/ME, but only if the codepage happens to be set to whatever
        + * we're inputting. */
        +
        + ret = pImmGetCompositionStringA(hIMC, GCS, NULL, 0);
        + if (ret <= 0)
        + return NULL; /* empty or error */
        else
        {
        ! char_u *buf;

        + buf = (char_u *) alloc(ret);
        + if (!buf)
        + return NULL;
        + pImmGetCompositionStringA(hIMC, GCS, buf, ret);

        ! /* convert from codepage to UCS-2 */
        ! wbuf = (unsigned short *)string_convert(&ime_conv_cp, buf, &ret);
        ! vim_free(buf);
        ! *len = ret / sizeof(unsigned short); /* char_u -> wchar */
        ! }
        ! return wbuf;
        }

        ! /*
        ! * void GetResultStr()
        ! *
        ! * This handles WM_IME_COMPOSITION with GCS_RESULTSTR flag on.
        ! * get complete composition string
        ! */
        ! static char_u *
        ! GetResultStr(HWND hwnd, int GCS)
        {
        ! DWORD dwBufLen; /* Stogare for len. of composition str. */
        ! HIMC hIMC; /* Input context handle. */
        ! unsigned short *buf = NULL;
        ! char *convbuf = NULL;

        if (!pImmGetContext || !(hIMC = pImmGetContext(hwnd)))
        ! return NULL;

        ! /* Reads in the composition string. */
        ! buf = GetCompositionString_inUCS2(hIMC, GCS, &dwBufLen);
        ! if (!buf)
        ! return NULL;

        ! convbuf = string_convert(&ime_conv, (unsigned char *) buf, &dwBufLen);
        pImmReleaseContext(hwnd, hIMC);
        + vim_free(buf);
        + return convbuf;
        }

        ! static char_u *
        ImeGetTempComposition(void)
        {
        if (bInComposition == TRUE)
        ***************
        *** 1459,1466 ****
        {
        pImmGetConversionStatus(hImc, &dwConvMode, &dwSentMode);
        pImmReleaseContext(s_hwnd, hImc);
        ! if ((dwConvMode & IME_CMODE_NATIVE))
        ! return lpCompStr;
        }
        }
        return NULL;
        --- 1529,1537 ----
        {
        pImmGetConversionStatus(hImc, &dwConvMode, &dwSentMode);
        pImmReleaseContext(s_hwnd, hImc);
        ! if ((dwConvMode & IME_CMODE_NATIVE)) {
        ! return GetResultStr(s_hwnd, GCS_COMPSTR);
        ! }
        }
        }
        return NULL;
        ***************
        *** 1668,1674 ****
        HBRUSH hbr;
        RECT rc;
        #ifdef FEAT_MBYTE_IME
        ! char *szComp;
        #endif

        if (!(flags & DRAW_TRANSP))
        --- 1739,1745 ----
        HBRUSH hbr;
        RECT rc;
        #ifdef FEAT_MBYTE_IME
        ! char_u *szComp;
        #endif

        if (!(flags & DRAW_TRANSP))
        ***************
        *** 1763,1772 ****
        {
        /* draw an incomplete composition character (korean) */
        if (len == 1 && blink_state == BLINK_ON
        ! && (szComp = ImeGetTempComposition()) != NULL) // hangul
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, szComp, 2, padding, TRUE);
        ! else
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, (char *)text, len, padding, FALSE);
        }
        --- 1834,1844 ----
        {
        /* draw an incomplete composition character (korean) */
        if (len == 1 && blink_state == BLINK_ON
        ! && (szComp = ImeGetTempComposition()) != NULL) { // hangul
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, szComp, 2, padding, TRUE);
        ! vim_free(szComp);
        ! } else
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, (char *)text, len, padding, FALSE);
        }
        ***************
        *** 3334,3342 ****
        hLibImm = LoadLibrary("imm32.dll");
        if (hLibImm == NULL)
        return;
        ! if ((*((FARPROC*)&pImmGetCompositionString)
        = GetProcAddress(hLibImm, "ImmGetCompositionStringA")))
        nImmFunc++;
        if ((*((FARPROC*)&pImmGetContext)
        = GetProcAddress(hLibImm, "ImmGetContext")))
        nImmFunc++;
        --- 3406,3417 ----
        hLibImm = LoadLibrary("imm32.dll");
        if (hLibImm == NULL)
        return;
        ! if ((*((FARPROC*)&pImmGetCompositionStringA)
        = GetProcAddress(hLibImm, "ImmGetCompositionStringA")))
        nImmFunc++;
        + if ((*((FARPROC*)&pImmGetCompositionStringW)
        + = GetProcAddress(hLibImm, "ImmGetCompositionStringW")))
        + nImmFunc++;
        if ((*((FARPROC*)&pImmGetContext)
        = GetProcAddress(hLibImm, "ImmGetContext")))
        nImmFunc++;
        ***************
        *** 3362,3368 ****
        = GetProcAddress(hLibImm, "ImmGetConversionStatus")))
        nImmFunc++;

        ! if (nImmFunc != 9)
        {
        FreeLibrary(hLibImm);
        hLibImm = NULL;
        --- 3437,3443 ----
        = GetProcAddress(hLibImm, "ImmGetConversionStatus")))
        nImmFunc++;

        ! if (nImmFunc != 10)
        {
        FreeLibrary(hLibImm);
        hLibImm = NULL;
      Your message has been successfully submitted and would be delivered to recipients shortly.