Loading ...
Sorry, an error occurred while loading the content.

Re: win2k IME, multibyte/width characters, etc.

Expand Messages
  • Glenn F. Maynard
    ... Again, Win32 doesn t seem to have any concept of unicode fonts, and the only fonts that support Japanese are extremely ugly in English. ... Perhaps; I ll
    Message 1 of 54 , Sep 24, 2001
    • 0 Attachment
      On Mon, Sep 24, 2001 at 01:08:24PM +0200, Bram Moolenaar wrote:
      > That should work when you are using one language. However, if you are
      > using different languages in Unicode, you really need a Unicode font.

      Again, Win32 doesn't seem to have any concept of unicode fonts, and the
      only fonts that support Japanese are extremely ugly in English.

      > Fixing those artifacts isn't that easy. You could end up drawing every
      > character separately, which is very slow. But it does allow using any
      > font (also proportional fonts). The character spacing will look ugly
      > though.

      Perhaps; I'll dig into it. The character spacing shouldn't look ugly,
      at least with Japanese fonts (in fact, when it was drawing correctly,
      it looked just fine with vastly differing point sizes and spacing).

      Anyhow, this is neither bug-related nor multibyte-related; I'll bring
      this up again (on vim-devel) if I come up with anything useful.

      > I don't think I have seen this patch separately. And my fonts look OK,
      > thus the problem isn't that generic.

      It is. Read the message; there are a couple fairly substantial bugs in font
      loading in Windows, particularly with spaces in font names. Try

      :set guifont=MS_Mincho:h10:cANSI
      (in the GUI, not in gvimrc)

      "Invalid font(s)". Try

      :set guifont=MS\ Mincho:h10:cANSI

      to work around the space/underscore bug, and it'll find *a* font, but it'll
      be some Korean font, not MS Mincho. (What you get is probably system-
      dependent.)

      Attached it; it got munged last time I posted it. (More details of this
      are in the other two messages on it; they're both included--one in quotes--
      in the post with the subject "fonts".)

      --
      Glenn Maynard
    • Muraoka Taro
      Sorry, I forgot to attach that two files. ... diff -cr src.orig/mbyte.c src/mbyte.c ... *************** ... if (enc_utf8 && !option_was_set((char_u *) fencs ))
      Message 54 of 54 , Oct 25, 2001
      • 0 Attachment
        Sorry, I forgot to attach that two files.

        > I had found two problems about Glenn's latest patch for IME.
        >
        > 1. In mbyte.c FEAT_WINDOWS is used for instead of WIN32.
        > It cause touble when compiling on non-windows platform.
        > Check attached ime_fix1.diff.
        > 2. I got report of strange behavior from an user of old IME (it is called SKK).
        > I had checked this problem.
        > SKK send WM_IME_COMPOSITION message when 'k' typed
        > without any composition string.
        > It cause unexpected two white space insertion.
        > I had fixed this.
        > Please check _OnImeComposition() and GetResultStr() in ime_fix2.diff.
        >
        > Regards.
        > ----
        > Muraoka Taro <koron@...>
        >

        ----ime_fix1.diff
        diff -cr src.orig/mbyte.c src/mbyte.c
        *** src.orig/mbyte.c Sun Oct 21 20:30:38 2001
        --- src/mbyte.c Sun Oct 21 20:30:33 2001
        ***************
        *** 572,577 ****
        --- 572,591 ----
        if (enc_utf8 && !option_was_set((char_u *)"fencs"))
        set_string_option_direct((char_u *)"fencs", -1,
        (char_u *)"ucs-bom,utf-8,latin1", OPT_FREE);
        + #ifdef FEAT_MBYTE_IME
        + # ifdef USE_ICONV
        + ime_conv.vc_fd = (iconv_t)-1;
        + # endif
        + convert_setup(&ime_conv, "ucs-2", p_enc);
        + #endif
        +
        + #ifdef FEAT_MBYTE_IME
        + # ifdef USE_ICONV
        + ime_conv_cp.vc_fd = (iconv_t)-1;
        + # endif
        + ime_conv_cp.vc_type = CONV_CODEPAGE;
        + ime_conv_cp.vc_factor = 2; /* we don't really know anything about the codepage */
        + #endif

        #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
        /* GNU gettext 0.10.37 supports this feature: set the codeset used for
        ***************
        *** 3539,3566 ****
        add_to_input_buf(str, 3);
        }

        - /*
        - * Add "str[len]" to the input buffer while escaping CSI bytes.
        - */
        - static void
        - add_to_input_buf_csi(char_u *str, int len)
        - {
        - int i;
        - char_u buf[2];
        -
        - for (i = 0; i < len; ++i)
        - {
        - add_to_input_buf(str + i, 1);
        - if (str[i] == CSI)
        - {
        - /* Turn CSI into K_CSI. */
        - buf[0] = KS_EXTRA;
        - buf[1] = (int)KE_CSI;
        - add_to_input_buf(buf, 2);
        - }
        - }
        - }
        -
        static GSList *key_press_event_queue = NULL;
        static int preedit_buf_len = 0;
        static gboolean processing_queued_event = FALSE;
        --- 3553,3558 ----
        ***************
        *** 3897,3902 ****
        --- 3889,3908 ----
        /* Internal utf-8 -> latin1 conversion. */
        vcp->vc_type = CONV_TO_LATIN1;
        }
        + #ifdef WIN32
        + /* Win32-specific UTF-16 -> DBCS conversion, for the IME,
        + * so we don't need iconv ... */
        + else if((from_prop & ENC_UNICODE) && (from_prop & ENC_2BYTE) && (to_prop & ENC_DBCS))
        + {
        + vcp->vc_type = CONV_DBCS;
        + vcp->vc_factor = 2; /* up to twice as long */
        + vcp->dbcs = atoi(to + 2);
        + } else if((from_prop & ENC_UNICODE) && (from_prop & ENC_2BYTE) && (to_prop & ENC_UNICODE)) {
        + vcp->vc_type = CONV_DBCS;
        + vcp->vc_factor = 2; /* up to twice as long */
        + vcp->dbcs = CP_UTF8;
        + }
        + #endif
        # ifdef USE_ICONV
        else
        {
        ***************
        *** 4023,4029 ****
        --- 4029,4077 ----
        retval = iconv_string(vcp->vc_fd, ptr, len);
        if (retval != NULL && lenp != NULL)
        *lenp = (int)STRLEN(retval);
        + break;
        # endif
        + # ifdef WIN32
        + case CONV_DBCS: /* UTF-16 -> dbcs or UTF8 */
        + {
        + int retlen;
        + if (!lenp)
        + len /= sizeof(unsigned short);
        +
        + retlen = WideCharToMultiByte(vcp->dbcs, 0,
        + (const unsigned short *)ptr, len, 0, 0, 0, 0);
        +
        + retval = alloc(retlen + 1);
        + if (retval == NULL)
        + break;
        +
        + WideCharToMultiByte(vcp->dbcs, 0,
        + (const unsigned short *) ptr, len, retval, retlen, 0, 0);
        +
        + retval[retlen] = NUL;
        + if (lenp != NULL)
        + *lenp = retlen;
        + break;
        + }
        + case CONV_CODEPAGE: /* current codepage -> ucs-2 */
        + {
        + int retlen;
        + retlen = MultiByteToWideChar(GetACP(), 0, ptr, len,
        + 0, 0);
        +
        + retval = alloc(sizeof(unsigned short) * retlen);
        + if (retval == NULL)
        + break;
        +
        + MultiByteToWideChar(GetACP(), 0, ptr, len,
        + (unsigned short *) retval, retlen);
        +
        + if (lenp != NULL)
        + *lenp = retlen * sizeof(unsigned short); /* number of shorts -> buffer size */
        + }
        +
        + # endif
        +
        }

        return retval;


        ----ime_fix2.diff
        diff -cr src.orig/gui_w32.c src/gui_w32.c
        *** src.orig/gui_w32.c Wed Sep 19 21:43:49 2001
        --- src/gui_w32.c Tue Oct 2 17:31:11 2001
        ***************
        *** 210,215 ****
        --- 210,219 ----
        static int get_toolbar_bitmap(vimmenu_T *menu);
        #endif

        + #ifdef FEAT_MBYTE_IME
        + static LRESULT _OnImeComposition(HWND hwnd, WPARAM dbcs, LPARAM param);
        + static char_u *GetResultStr(HWND hwnd, int GCS);
        + #endif
        #if defined(FEAT_MBYTE_IME) && defined(DYNAMIC_IME)
        # ifdef NOIME
        typedef struct tagCOMPOSITIONFORM {
        ***************
        *** 221,227 ****
        # endif

        HINSTANCE hLibImm = NULL;
        ! LONG (WINAPI *pImmGetCompositionString)(HIMC, DWORD, LPVOID, DWORD);
        HIMC (WINAPI *pImmGetContext)(HWND);
        BOOL (WINAPI *pImmReleaseContext)(HWND, HIMC);
        BOOL (WINAPI *pImmGetOpenStatus)(HIMC);
        --- 225,232 ----
        # endif

        HINSTANCE hLibImm = NULL;
        ! LONG (WINAPI *pImmGetCompositionStringA)(HIMC, DWORD, LPVOID, DWORD);
        ! LONG (WINAPI *pImmGetCompositionStringW)(HIMC, DWORD, LPVOID, DWORD);
        HIMC (WINAPI *pImmGetContext)(HWND);
        BOOL (WINAPI *pImmReleaseContext)(HWND, HIMC);
        BOOL (WINAPI *pImmGetOpenStatus)(HIMC);
        ***************
        *** 232,238 ****
        BOOL (WINAPI *pImmGetConversionStatus)(HIMC, LPDWORD, LPDWORD);
        static void dyn_imm_load(void);
        #else
        ! # define pImmGetCompositionString ImmGetCompositionStringA
        # define pImmGetContext ImmGetContext
        # define pImmReleaseContext ImmReleaseContext
        # define pImmGetOpenStatus ImmGetOpenStatus
        --- 237,244 ----
        BOOL (WINAPI *pImmGetConversionStatus)(HIMC, LPDWORD, LPDWORD);
        static void dyn_imm_load(void);
        #else
        ! # define pImmGetCompositionStringA ImmGetCompositionStringA
        ! # define pImmGetCompositionStringW ImmGetCompositionStringW
        # define pImmGetContext ImmGetContext
        # define pImmReleaseContext ImmReleaseContext
        # define pImmGetOpenStatus ImmGetOpenStatus
        ***************
        *** 790,795 ****
        --- 841,852 ----
        case WM_IME_NOTIFY:
        if (!_OnImeNotify(hwnd, (DWORD)wParam, (DWORD)lParam))
        return DefWindowProc(hwnd, uMsg, wParam, lParam);
        + break;
        +
        + case WM_IME_COMPOSITION:
        + if (!_OnImeComposition(hwnd, wParam, lParam))
        + return DefWindowProc(hwnd, uMsg, wParam, lParam);
        + break;
        #endif

        default:
        ***************
        *** 1292,1298 ****
        #include <ime.h>
        #include <imm.h>

        - static char lpCompStr[100]; // Pointer to composition str.
        static BOOL bInComposition=FALSE;

        /*
        --- 1352,1357 ----
        ***************
        *** 1356,1453 ****
        }
        }
        gui_update_cursor(TRUE, FALSE);
        ! lResult = 1;
        break;
        }
        pImmReleaseContext(hWnd, hImc);
        return lResult;
        }

        ! /* get composition string from WIN_IME */
        ! static void
        ! GetCompositionStr(HWND hwnd, LPARAM CompFlag)
        {
        ! DWORD dwBufLen; // Stogare for len. of composition str
        ! HIMC hIMC; // Input context handle.
        !
        ! // If fail to get input context handle then do nothing.
        ! // Applications should call ImmGetContext API to get
        ! // input context handle.

        ! if (!pImmGetContext || !(hIMC = pImmGetContext(hwnd)))
        ! return;

        ! // Determines how much memory space to store the composition string.
        ! // Applications should call ImmGetCompositionString with
        ! // GCS_COMPSTR flag on, buffer length zero, to get the bullfer
        ! // length.

        ! if ((dwBufLen = pImmGetCompositionString(hIMC, GCS_COMPSTR,
        ! (void FAR*)NULL, 0l)) < 0)
        ! goto exit2;

        ! if (dwBufLen > 99)
        ! goto exit2;

        ! // Reads in the composition string.
        ! if ( dwBufLen != 0 )
        {
        ! pImmGetCompositionString(hIMC, GCS_COMPSTR, lpCompStr, dwBufLen);
        ! lpCompStr[dwBufLen] = 0;
        }
        else
        {
        ! strcpy(lpCompStr, " ");
        ! dwBufLen = 2;
        ! }
        !
        ! // Display new composition chars.
        ! DisplayCompStringOpaque(lpCompStr, dwBufLen);


        ! exit2:
        ! pImmReleaseContext(hwnd, hIMC);
        }

        !
        ! // void GetResultStr()
        ! //
        ! // This handles WM_IME_COMPOSITION with GCS_RESULTSTR flag on.
        ! //
        ! // get complete composition string
        !
        ! static void
        ! GetResultStr(HWND hwnd)
        {
        ! DWORD dwBufLen; // Storage for length of result str.
        ! HIMC hIMC; // Input context handle.

        - // If fail to get input context handle then do nothing.
        if (!pImmGetContext || !(hIMC = pImmGetContext(hwnd)))
        ! return;
        !
        ! // Determines how much memory space to store the result string.
        ! // Applications should call ImmGetCompositionString with
        ! // GCS_RESULTSTR flag on, buffer length zero, to get the bullfer
        ! // length.
        ! if ((dwBufLen = pImmGetCompositionString(hIMC, GCS_RESULTSTR,
        ! (void FAR *)NULL, (DWORD) 0)) <= 0)
        ! goto exit2;
        !
        ! if (dwBufLen > 99)
        ! goto exit2;
        !
        ! // Reads in the result string.
        ! pImmGetCompositionString(hIMC, GCS_RESULTSTR, lpCompStr, dwBufLen);

        ! // Displays the result string.
        ! DisplayCompStringOpaque(lpCompStr, dwBufLen);

        ! exit2:
        pImmReleaseContext(hwnd, hIMC);
        }

        ! static char *
        ImeGetTempComposition(void)
        {
        if (bInComposition == TRUE)
        --- 1415,1523 ----
        }
        }
        gui_update_cursor(TRUE, FALSE);
        ! lResult = 0;
        break;
        }
        pImmReleaseContext(hWnd, hImc);
        return lResult;
        }

        ! static LRESULT
        ! _OnImeComposition(HWND hwnd, WPARAM dbcs, LPARAM param)
        {
        ! char_u *ret;
        ! if ((param & GCS_RESULTSTR) == 0) /* Composition unfinished. */
        ! return 0;

        ! if (ret = GetResultStr(hwnd, GCS_RESULTSTR))
        ! {
        ! add_to_input_buf_csi(ret, strlen(ret));
        ! vim_free(ret);
        ! return 1;
        ! }
        ! else
        ! return 0;
        ! }

        ! /* get the currnet composition string, in UCS-2; len is the number of
        ! * Unicode characters */
        ! unsigned short *
        ! GetCompositionString_inUCS2(HIMC hIMC, DWORD GCS, int *len)
        ! {
        ! LONG ret;
        ! unsigned short *wbuf = NULL;

        ! if (!pImmGetContext)
        ! return NULL; /* no imm32.dll */

        ! /* Try Unicode; this'll always work on NT regardless of codepage. */
        ! ret = pImmGetCompositionStringW(hIMC, GCS, NULL, 0);
        ! if (ret == 0)
        ! return NULL; /* empty */

        ! if (ret > 0)
        {
        ! wbuf = (unsigned short *) alloc(ret * sizeof(unsigned short));
        ! if(!wbuf) return NULL;
        !
        ! pImmGetCompositionStringW(hIMC, GCS, wbuf, ret);
        ! *len = ret / sizeof(unsigned short); /* char -> wchar */
        ! return wbuf;
        }
        +
        + /* ret < 0; we got an error, so try the ANSI version. This'll work
        + * on 9x/ME, but only if the codepage happens to be set to whatever
        + * we're inputting. */
        +
        + ret = pImmGetCompositionStringA(hIMC, GCS, NULL, 0);
        + if (ret <= 0)
        + return NULL; /* empty or error */
        else
        {
        ! char_u *buf;

        + buf = (char_u *) alloc(ret);
        + if (!buf)
        + return NULL;
        + pImmGetCompositionStringA(hIMC, GCS, buf, ret);

        ! /* convert from codepage to UCS-2 */
        ! wbuf = (unsigned short *)string_convert(&ime_conv_cp, buf, &ret);
        ! vim_free(buf);
        ! *len = ret / sizeof(unsigned short); /* char_u -> wchar */
        ! }
        ! return wbuf;
        }

        ! /*
        ! * void GetResultStr()
        ! *
        ! * This handles WM_IME_COMPOSITION with GCS_RESULTSTR flag on.
        ! * get complete composition string
        ! */
        ! static char_u *
        ! GetResultStr(HWND hwnd, int GCS)
        {
        ! DWORD dwBufLen; /* Stogare for len. of composition str. */
        ! HIMC hIMC; /* Input context handle. */
        ! unsigned short *buf = NULL;
        ! char *convbuf = NULL;

        if (!pImmGetContext || !(hIMC = pImmGetContext(hwnd)))
        ! return NULL;

        ! /* Reads in the composition string. */
        ! buf = GetCompositionString_inUCS2(hIMC, GCS, &dwBufLen);
        ! if (!buf)
        ! return NULL;

        ! convbuf = string_convert(&ime_conv, (unsigned char *) buf, &dwBufLen);
        pImmReleaseContext(hwnd, hIMC);
        + vim_free(buf);
        + return convbuf;
        }

        ! static char_u *
        ImeGetTempComposition(void)
        {
        if (bInComposition == TRUE)
        ***************
        *** 1459,1466 ****
        {
        pImmGetConversionStatus(hImc, &dwConvMode, &dwSentMode);
        pImmReleaseContext(s_hwnd, hImc);
        ! if ((dwConvMode & IME_CMODE_NATIVE))
        ! return lpCompStr;
        }
        }
        return NULL;
        --- 1529,1537 ----
        {
        pImmGetConversionStatus(hImc, &dwConvMode, &dwSentMode);
        pImmReleaseContext(s_hwnd, hImc);
        ! if ((dwConvMode & IME_CMODE_NATIVE)) {
        ! return GetResultStr(s_hwnd, GCS_COMPSTR);
        ! }
        }
        }
        return NULL;
        ***************
        *** 1668,1674 ****
        HBRUSH hbr;
        RECT rc;
        #ifdef FEAT_MBYTE_IME
        ! char *szComp;
        #endif

        if (!(flags & DRAW_TRANSP))
        --- 1739,1745 ----
        HBRUSH hbr;
        RECT rc;
        #ifdef FEAT_MBYTE_IME
        ! char_u *szComp;
        #endif

        if (!(flags & DRAW_TRANSP))
        ***************
        *** 1763,1772 ****
        {
        /* draw an incomplete composition character (korean) */
        if (len == 1 && blink_state == BLINK_ON
        ! && (szComp = ImeGetTempComposition()) != NULL) // hangul
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, szComp, 2, padding, TRUE);
        ! else
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, (char *)text, len, padding, FALSE);
        }
        --- 1834,1844 ----
        {
        /* draw an incomplete composition character (korean) */
        if (len == 1 && blink_state == BLINK_ON
        ! && (szComp = ImeGetTempComposition()) != NULL) { // hangul
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, szComp, 2, padding, TRUE);
        ! vim_free(szComp);
        ! } else
        HanExtTextOut(s_hdc, TEXT_X(col), TEXT_Y(row),
        foptions, pcliprect, (char *)text, len, padding, FALSE);
        }
        ***************
        *** 3334,3342 ****
        hLibImm = LoadLibrary("imm32.dll");
        if (hLibImm == NULL)
        return;
        ! if ((*((FARPROC*)&pImmGetCompositionString)
        = GetProcAddress(hLibImm, "ImmGetCompositionStringA")))
        nImmFunc++;
        if ((*((FARPROC*)&pImmGetContext)
        = GetProcAddress(hLibImm, "ImmGetContext")))
        nImmFunc++;
        --- 3406,3417 ----
        hLibImm = LoadLibrary("imm32.dll");
        if (hLibImm == NULL)
        return;
        ! if ((*((FARPROC*)&pImmGetCompositionStringA)
        = GetProcAddress(hLibImm, "ImmGetCompositionStringA")))
        nImmFunc++;
        + if ((*((FARPROC*)&pImmGetCompositionStringW)
        + = GetProcAddress(hLibImm, "ImmGetCompositionStringW")))
        + nImmFunc++;
        if ((*((FARPROC*)&pImmGetContext)
        = GetProcAddress(hLibImm, "ImmGetContext")))
        nImmFunc++;
        ***************
        *** 3362,3368 ****
        = GetProcAddress(hLibImm, "ImmGetConversionStatus")))
        nImmFunc++;

        ! if (nImmFunc != 9)
        {
        FreeLibrary(hLibImm);
        hLibImm = NULL;
        --- 3437,3443 ----
        = GetProcAddress(hLibImm, "ImmGetConversionStatus")))
        nImmFunc++;

        ! if (nImmFunc != 10)
        {
        FreeLibrary(hLibImm);
        hLibImm = NULL;
      Your message has been successfully submitted and would be delivered to recipients shortly.