Loading ...
Sorry, an error occurred while loading the content.

win32 gvim: font width problem

Expand Messages
  • Glenn Maynard
    This is more of a font problem than a multibyte problem, so I sent this here instead of vim-multibyte. Load Win32 gvim with MS Mincho (Japanese font), set
    Message 1 of 12 , Jan 30, 2002
    View Source
    • 0 Attachment
      This is more of a font problem than a multibyte problem, so I sent this
      here instead of vim-multibyte.

      Load Win32 gvim with MS Mincho (Japanese font), set encoding=utf8, and put
      some degree symbols (°) among some other text. Things mess up. It
      looks licke this is a double-width glyph in this font, where it's
      expected to be single-width.

      Putty simply ignores the fact that it's double-width and prints it as
      single-width anyway. This works fine for this particular glyph, since
      there's nothing actually on the right half.

      This throws it for a loop:

      ++,,--..00II♦♦▒▒°°±±**┘┘┐┐┌┌└└┼┼----──----├├┤┤┴┴┬┬││≤≤≥≥ππ≠≠££··

      (That doesn't look quite right in Putty: the not-equals-to and pi
      characters are odd widths, too. It doesn't break, though.)

      --
      Glenn Maynard
    • Bram Moolenaar
      ... This probably means it s a faulty font. Vim doesn t take care of that, because it would slow down displaying text. ... Do you mean Vim hangs? -- Nobody
      Message 2 of 12 , Jan 31, 2002
      View Source
      • 0 Attachment
        Glenn Maynard wrote:

        > This is more of a font problem than a multibyte problem, so I sent this
        > here instead of vim-multibyte.
        >
        > Load Win32 gvim with MS Mincho (Japanese font), set encoding=utf8, and put
        > some degree symbols (°) among some other text. Things mess up. It
        > looks licke this is a double-width glyph in this font, where it's
        > expected to be single-width.
        >
        > Putty simply ignores the fact that it's double-width and prints it as
        > single-width anyway. This works fine for this particular glyph, since
        > there's nothing actually on the right half.

        This probably means it's a faulty font. Vim doesn't take care of that,
        because it would slow down displaying text.

        > This throws it for a loop:
        >
        > ++,,--..00II♦♦▒▒°°±±**┘┘┐┐┌┌└└┼┼----──----├├┤┤┴┴┬┬││≤≤≥≥ππ≠≠££··

        Do you mean Vim hangs?

        --
        Nobody will ever need more than 640 kB RAM.
        -- Bill Gates, 1983
        Windows 98 requires 16 MB RAM.
        -- Bill Gates, 1999
        Logical conclusion: Nobody will ever need Windows 98.

        /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
        ((( Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim )))
        \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
      • Glenn Maynard
        ... Well, it s one of the basic system fonts, and there aren t many options for Japanese text. This isn t a case where I can simply replace the font--if it s
        Message 3 of 12 , Jan 31, 2002
        View Source
        • 0 Attachment
          On Thu, Jan 31, 2002 at 11:10:29AM +0100, Bram Moolenaar wrote:
          > This probably means it's a faulty font. Vim doesn't take care of that,
          > because it would slow down displaying text.

          Well, it's one of the basic system fonts, and there aren't many options
          for Japanese text. This isn't a case where I can simply replace the font--if
          it's a font problem, it's a system one that's not going away, so it
          really needs a workaround.

          > > This throws it for a loop:
          > >
          > > ++,,--..00II♦♦▒▒°°±±**┘┘┐┐┌┌└└┼┼----──----├├┤┤┴┴┬┬││≤≤≥≥ππ≠≠££··
          >
          > Do you mean Vim hangs?

          Text becomes misaligned; it's reminiscent of an ncurses app losing track
          of the cursor position.

          --
          Glenn Maynard
        • Bram Moolenaar
          ... What do you expect Vim to do? Give you a warning that this font is faulty? Detect that a faulty font is used (how?) and switch to displaying each
          Message 4 of 12 , Jan 31, 2002
          View Source
          • 0 Attachment
            Glenn Maynard wrote:

            > On Thu, Jan 31, 2002 at 11:10:29AM +0100, Bram Moolenaar wrote:
            > > This probably means it's a faulty font. Vim doesn't take care of that,
            > > because it would slow down displaying text.
            >
            > Well, it's one of the basic system fonts, and there aren't many options
            > for Japanese text. This isn't a case where I can simply replace the font--if
            > it's a font problem, it's a system one that's not going away, so it
            > really needs a workaround.

            What do you expect Vim to do? Give you a warning that this font is
            faulty? Detect that a faulty font is used (how?) and switch to
            displaying each character separately? This is especially tricky when
            several versions of the same font exist (e.g., when it's fixed by a
            service pack).

            > > > This throws it for a loop:
            > > >
            > > > ++,,--..00II♦♦▒▒°°±±**┘┘┐┐┌┌└└┼┼----──----├├┤┤┴┴┬┬││≤≤≥≥ππ≠≠££··
            > >
            > > Do you mean Vim hangs?
            >
            > Text becomes misaligned; it's reminiscent of an ncurses app losing track
            > of the cursor position.

            When I look at this on my FreeBSD system with a standard Unicode font I
            see only single-width characters. Where do you see a problem?

            --
            hundred-and-one symptoms of being an internet addict:
            127. You bring your laptop and cellular phone to church.

            /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
            ((( Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim )))
            \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
          • Glenn Maynard
            ... I believe Putty displays each character separately. I don t know how much of a speed hit this is; Putty is extremely fast on all systems I ve used it on,
            Message 5 of 12 , Jan 31, 2002
            View Source
            • 0 Attachment
              On Thu, Jan 31, 2002 at 08:46:35PM +0100, Bram Moolenaar wrote:
              > > Well, it's one of the basic system fonts, and there aren't many options
              > > for Japanese text. This isn't a case where I can simply replace the font--if
              > > it's a font problem, it's a system one that's not going away, so it
              > > really needs a workaround.
              >
              > What do you expect Vim to do? Give you a warning that this font is
              > faulty? Detect that a faulty font is used (how?) and switch to
              > displaying each character separately? This is especially tricky when
              > several versions of the same font exist (e.g., when it's fixed by a
              > service pack).

              I believe Putty displays each character separately. I don't know how much
              of a speed hit this is; Putty is extremely fast on all systems I've used it
              on, but I've never used it on a really old system.

              In theory, you could test it: look at each glyph and see if the glyph
              widths are really as they should be (either the font's width or a
              multiple of.) I havn't tried this.

              Here's a suggestion: if you know which characters are of the incorrect
              width, you can render all valid characters together, and just render
              those characters individually. ("abcdπefg"; render "abcd", "π", "efg".)

              If done right, this would also make it possible to have multiple fonts for
              different Unicode ranges in Windows, too, which would be extremely nice to
              have. (I think you rejected this earlier for the same reason: you don't
              want to render character-by-character. With the above optimization, it
              should have no affect for people that don't use it, and little for those
              who do.)

              > When I look at this on my FreeBSD system with a standard Unicode font I
              > see only single-width characters. Where do you see a problem?

              I'll take a few screenshots; it's difficult to explain. I've taken out
              everything but the pi symbol, so it's easier to tell what's happening.

              http://zewt.org/~glenn/vim1.jpg
              http://zewt.org/~glenn/vim2.jpg

              The pi character has an incorrect width. There's no clean way to get a
              proper looking pi symbol here (the font just doesn't have one), but
              currently it makes the file difficult to even edit.

              --
              Glenn Maynard
            • Bram Moolenaar
              ... I would guess that estimating the glyph width of each character would be a slowdown already. This would need to be tried out. Caching the resulting
              Message 6 of 12 , Feb 1, 2002
              View Source
              • 0 Attachment
                Glenn Maynard wrote:

                > On Thu, Jan 31, 2002 at 08:46:35PM +0100, Bram Moolenaar wrote:
                > > > Well, it's one of the basic system fonts, and there aren't many options
                > > > for Japanese text. This isn't a case where I can simply replace
                > > > the font--if it's a font problem, it's a system one that's not
                > > > going away, so it really needs a workaround.
                > >
                > > What do you expect Vim to do? Give you a warning that this font is
                > > faulty? Detect that a faulty font is used (how?) and switch to
                > > displaying each character separately? This is especially tricky when
                > > several versions of the same font exist (e.g., when it's fixed by a
                > > service pack).
                >
                > I believe Putty displays each character separately. I don't know how much
                > of a speed hit this is; Putty is extremely fast on all systems I've used it
                > on, but I've never used it on a really old system.
                >
                > In theory, you could test it: look at each glyph and see if the glyph
                > widths are really as they should be (either the font's width or a
                > multiple of.) I havn't tried this.
                >
                > Here's a suggestion: if you know which characters are of the incorrect
                > width, you can render all valid characters together, and just render
                > those characters individually. ("abcd$B&P(Befg"; render "abcd", "$B&P(B", "efg".)

                I would guess that estimating the glyph width of each character would be
                a slowdown already. This would need to be tried out. Caching the
                resulting values would help, but it does become a rather complicated
                workaround for a bug in a font.

                > If done right, this would also make it possible to have multiple fonts for
                > different Unicode ranges in Windows, too, which would be extremely nice to
                > have. (I think you rejected this earlier for the same reason: you don't
                > want to render character-by-character. With the above optimization, it
                > should have no affect for people that don't use it, and little for those
                > who do.)

                The solution for this would be to create a new font that includes the
                characters you like. Big advantage is that you can use it in many
                programs.

                > > When I look at this on my FreeBSD system with a standard Unicode font I
                > > see only single-width characters. Where do you see a problem?
                >
                > I'll take a few screenshots; it's difficult to explain. I've taken out
                > everything but the pi symbol, so it's easier to tell what's happening.
                >
                > http://zewt.org/~glenn/vim1.jpg
                > http://zewt.org/~glenn/vim2.jpg
                >
                > The pi character has an incorrect width. There's no clean way to get a
                > proper looking pi symbol here (the font just doesn't have one), but
                > currently it makes the file difficult to even edit.

                Is the pi character 0x03c0 (or 0x03a0)? Those are single width, thus
                it's definitely a font problem. Why don't you fix the font?

                --
                Why is it called "Windows"? "Gates" would be more appropriate...

                /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
                ((( Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim )))
                \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
              • Glenn Maynard
                ... It s a lot more worthwhile if it s also used to allow the below. ... I know of no program that ll let me mix and match glyphs from multiple TTF, TTC and
                Message 7 of 12 , Feb 1, 2002
                View Source
                • 0 Attachment
                  On Fri, Feb 01, 2002 at 10:29:18AM +0100, Bram Moolenaar wrote:
                  > I would guess that estimating the glyph width of each character would be
                  > a slowdown already. This would need to be tried out. Caching the
                  > resulting values would help, but it does become a rather complicated
                  > workaround for a bug in a font.

                  It's a lot more worthwhile if it's also used to allow the below.

                  > > If done right, this would also make it possible to have multiple fonts for
                  > > different Unicode ranges in Windows, too, which would be extremely nice to
                  > > have. (I think you rejected this earlier for the same reason: you don't
                  > > want to render character-by-character. With the above optimization, it
                  > > should have no affect for people that don't use it, and little for those
                  > > who do.)
                  >
                  > The solution for this would be to create a new font that includes the
                  > characters you like. Big advantage is that you can use it in many
                  > programs.

                  I know of no program that'll let me mix and match glyphs from multiple
                  TTF, TTC and raster fonts. (I've been looking for one for a long time,
                  for exactly this purpose.)

                  > Is the pi character 0x03c0 (or 0x03a0)? Those are single width, thus
                  > it's definitely a font problem. Why don't you fix the font?

                  (Actually, if you look closely, the rendered character is larger than
                  single-width, but smaller than double-width, so this isn't even a real
                  fixed-width font.)

                  The only time I've ever seen anything edit those fonts is to fix the old
                  backslash/yen symbol problem, and that was a hexedit. (And it doesn't
                  work on my system, since it was for NT4's TTFs; 2k uses TTCs, and the font
                  is probably slightly different, too.) Fixing this at the source would be
                  nice, but with MS software, that's usually impossible to do reliably.

                  This is the same as any bug in MS APIs: they aren't going to fix it,
                  since it's cheaper for them to make everyone else work around it.

                  --
                  Glenn Maynard
                • Muraoka Taro
                  I made an experimental patch. Attached will help you. ... Muraoka Taro
                  Message 8 of 12 , Feb 2, 2002
                  View Source
                  • 0 Attachment
                    I made an experimental patch.
                    Attached will help you.
                    ----
                    Muraoka Taro <koron@...>


                    > Putty simply ignores the fact that it's double-width and prints it as
                    > single-width anyway. This works fine for this particular glyph, since
                    > there's nothing actually on the right half.
                  • Glenn Maynard
                    (Minor: you could drop the charwidth_cache_initialized stuff. Static data is always initted to all 0 unless given otherwise; charwidth_cache[] will be all
                    Message 9 of 12 , Feb 2, 2002
                    View Source
                    • 0 Attachment
                      (Minor: you could drop the charwidth_cache_initialized stuff. Static data
                      is always initted to all 0 unless given otherwise; charwidth_cache[] will
                      be all null. This is guaranteed by C.)

                      On Sat, Feb 02, 2002 at 07:14:51PM +0900, Muraoka Taro wrote:
                      > I made an experimental patch.
                      > Attached will help you.

                      1: I don't think this is a double-width problem anymore. Look at
                      http://zewt.org/~glenn/pi.jpg; note that it doesn't realign properly.
                      The pi symbol isn't quite double-width, so I suspect it'll still end up
                      misaligned. (I havn't tried this yet; it's late and I don't have the
                      source handy.)

                      2: To operate "correctly", we should always output that character as
                      single-width; if the font is off, we should err to Unicode, not the
                      font. (That is, this fix may make it behave reasonably, but not
                      correctly.) This is a nitpick; I'd be personally willing to accept
                      mismatched character widths. (Even though it would, in theory, throw off
                      ASCII-formatted textfiles, it's not nearly as much of a problem as, say,
                      a terminal emulator mismatched with the underlying system.) You're
                      probably more familiar with this issue than I am--are most Japanese
                      users willing to accept minor width glitches like this?

                      If #1 is incorrect (it may be), then this is probably sufficient; I'm
                      not too concerned with #2 (exact text formatting isn't terribly
                      important to me, at least in cases this isolated), though as it'd still
                      be technically incorrect, I'd at least put #2 down as a low-pri todo.

                      Thanks.

                      --
                      Glenn Maynard
                    • Muraoka Taro
                      ... I saw that. But I think it is one of Windows BUGS. How point size of font did you use?. It may be 10 or 11. Please try 9 or 12, it should be aligned
                      Message 10 of 12 , Feb 2, 2002
                      View Source
                      • 0 Attachment
                        > 1: I don't think this is a double-width problem anymore. Look at
                        > http://zewt.org/~glenn/pi.jpg; note that it doesn't realign properly.
                        > The pi symbol isn't quite double-width, so I suspect it'll still end
                        > up misaligned. (I havn't tried this yet; it's late and I don't have
                        > the source handy.)

                        I saw that. But I think it is one of Windows BUGS. How point size of
                        font did you use?. It may be 10 or 11. Please try 9 or 12, it should
                        be aligned properly. (I think Windows can correctly show fix-width font
                        which multiple of 3 points size.)
                        ----
                        Muraoka Taro <koron@...>
                      • Bram Moolenaar
                        ... I m glad you made something for this. Does this take into account that Vim can use several fonts at the same time? E.g. for bold and other highlight
                        Message 11 of 12 , Feb 2, 2002
                        View Source
                        • 0 Attachment
                          Muraoka Taro wrote:

                          > I made an experimental patch.
                          > Attached will help you.

                          I'm glad you made something for this.

                          Does this take into account that Vim can use several fonts at the same
                          time? E.g. for bold and other highlight groups. These fonts should all
                          use the same font size, but since this problem is about size differences
                          the patch might need to handle wrong sizes anyway.

                          --
                          hundred-and-one symptoms of being an internet addict:
                          159. You get excited whenever discussing your hard drive.

                          /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
                          ((( Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim )))
                          \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
                        • Muraoka Taro
                          I wrote new patch. Please check it. ... From: To: Sent: Sunday, February 03, 2002 5:09 PM Subject: Re: win32 gvim:
                          Message 12 of 12 , Feb 3, 2002
                          View Source
                          • 0 Attachment
                            I wrote new patch.
                            Please check it.

                            ----- Original Message -----
                            From: <KGD03011@...>
                            To: <koron@...>
                            Sent: Sunday, February 03, 2002 5:09 PM
                            Subject: Re: win32 gvim: font width problem

                            (snip)
                            > But I've noticed these problems:
                            >
                            > 1. Starting VIM and changing fonts and font sizes is very slow
                            > (15 to 30 seconds).

                            Try new one. Cache became more clever than before.

                            > 2. At font-sizes 26 and 28 with Japanese fonts in utf-8, Vim
                            > seems to think that all characters, including kana, CJK etc, are
                            > half-width.

                            May be fixed. But there is another problem, Windows compress width of
                            drawing text when using big size font. As a result of this, some dust
                            will be shown.

                            > 3. In utf-8, an insertion of text that moves text to near the
                            > right margin of the window causes a crash. This does not happen
                            > with cp932.

                            I can't reproduce this.
                            ----
                            Muraoka Taro <koron@...>
                          Your message has been successfully submitted and would be delivered to recipients shortly.