Loading ...
Sorry, an error occurred while loading the content.

Re: CarbonVim encoding problem

Expand Messages
  • Sébastien Pierre
    Hi! ... I tried : set termencoding=8bit-MacRoman set encoding=latin-1 set fileencoding=latin-1 set fileencodings=utf-8,latin-1,8bit-MacRoman and set
    Message 1 of 14 , May 25, 2002
    • 0 Attachment
      Hi!

      Le vendredi 24 mai 2002, à 08:41 PM, Bram Moolenaar a écrit :

      > Perhaps 'termencoding' needs to be set?

      I tried :

      set termencoding=8bit-MacRoman
      set encoding=latin-1
      set fileencoding=latin-1
      set fileencodings=utf-8,latin-1,8bit-MacRoman

      and

      set termencoding=latin-1
      set encoding=8bit-MacRoman
      set fileencoding=latin-1
      set fileencodings=utf-8,latin-1,8bit-MacRoman

      and

      set termencoding=8bit-MacRoman
      set encoding=8bit-MacRoman
      set fileencoding=latin-1
      set fileencodings=utf-8,latin-1,8bit-MacRoman

      this last solution, even with an explicit fileencoding=latin-1 did not
      produce the expected <E9> charcater instead of the <8E> when pressing
      'é'.

      > Alternatively, you can set 'encoding' to "8bit-MacRoman". Don't know
      > how well this works though. You probably also need to set
      > 'fileencodings' to include "8bit-MacRoman" instead of "latin1".

      Removing the termencoding in my .vimrc did not change anything, removing
      latin-1 from the filencodings did nothing as well.

      >> To fix this, the keyboard input must be fixed to properly convert the
      >> input key code to vim internal keycode, the same thing being necessary
      >> for the character outputting. I don't think the change would be hard to
      >> make as this would be just like overloading the default behaviour of
      >> character reads and writes using the encoding translation tables.
      >
      > This is possible by setting 'termencoding', but conversion for 8-bit
      > characters usually means some characters will be dropped. Setting
      > 'encoding' might work better.

      If I understood well:

      - termencoding specifies the encoding in which keyboard input characters
      are mapped. For CarbonVim this is 8bit-MacRoman
      - encoding specifies the display encoding, which should also be
      8bit-MacRoman, but as shown before the encoding variable does not seem
      to have any effect, even when used without termencoding
      - fileencoding forces a specific encoding for the current file, trying
      to do the conversion between the encoding. This should be latin-1, as
      latin-1 is more likely to be read properly on other Unices than MacRoman
      - fileencodings lists the different encodings that are tried when
      reading a file, so this should be utf-8 first, which will most likely
      fail, then latin-1 and eventually MacRoman.

      I think the problem may be that CarbonVim is not linked with iconv, so
      that it does not actually do any conversion, but I must also admit that
      I still don't have a crystal clear understanding of the four above
      variables...

      Thanks for your help,

      -- Sébastien.


      --
      «Il vaut mieux suivre une voix stupide que l'on connait,
      plutôt qu'une intelligente que l'on ne connait pas.»
      <http://www.type-z.org> -- No comment on Jacques C.
    • Bram Moolenaar
      ... Putting both latin-1 and 8bit-MacRoman in fileencodings will always result in using latin-1, because there is no way to detect that a file isn t latin-1.
      Message 2 of 14 , May 25, 2002
      • 0 Attachment
        Sébastien Pierre wrote:

        > > Perhaps 'termencoding' needs to be set?
        >
        > I tried :
        >
        > set termencoding=8bit-MacRoman
        > set encoding=latin-1
        > set fileencoding=latin-1
        > set fileencodings=utf-8,latin-1,8bit-MacRoman

        Putting both latin-1 and 8bit-MacRoman in 'fileencodings' will always
        result in using latin-1, because there is no way to detect that a file
        isn't latin-1. Setting 'fileencoding' has no effect, it will be changed
        to one of the entries in 'fileencodings' as soon as you load a file.

        > set termencoding=8bit-MacRoman
        > set encoding=8bit-MacRoman
        > set fileencoding=latin-1
        > set fileencodings=utf-8,latin-1,8bit-MacRoman
        >
        > this last solution, even with an explicit fileencoding=latin-1 did not
        > produce the expected <E9> charcater instead of the <8E> when pressing
        > 'é'.

        When 'termencoding' is equal to 'encoding' or empty no conversion is
        done.

        > > Alternatively, you can set 'encoding' to "8bit-MacRoman". Don't know
        > > how well this works though. You probably also need to set
        > > 'fileencodings' to include "8bit-MacRoman" instead of "latin1".
        >
        > Removing the termencoding in my .vimrc did not change anything, removing
        > latin-1 from the filencodings did nothing as well.

        Did you check the value of 'fileencoding' after loading a file?

        > If I understood well:
        >
        > - termencoding specifies the encoding in which keyboard input characters
        > are mapped. For CarbonVim this is 8bit-MacRoman

        Yes, and for terminal output it also specifies conversion for displayed
        characters.

        > - encoding specifies the display encoding, which should also be
        > 8bit-MacRoman, but as shown before the encoding variable does not seem
        > to have any effect, even when used without termencoding

        'encoding' specifies the internal encoding. For 8-bit encodings the
        name isn't important for the internal stuff, except that it's used for
        conversions. For the GUI there is no conversion for displaying, thus
        'encoding' also sets the encoding for displaying. You need to use the
        right 'guifont' to display characters for 'encoding'.

        > - fileencoding forces a specific encoding for the current file, trying
        > to do the conversion between the encoding. This should be latin-1, as
        > latin-1 is more likely to be read properly on other Unices than MacRoman

        Only for writing. When reading 'fileencoding' is set from the values in
        'termencodings'. Or it's made empty when the file can't be read in any
        of these (which means 'encoding' is used). Unless you use ":edit
        ++enc=name file", then it's forced to use "name".

        > - fileencodings lists the different encodings that are tried when
        > reading a file, so this should be utf-8 first, which will most likely
        > fail, then latin-1 and eventually MacRoman.

        As said above, if you put latin-1 first, MacRoman will never be used,
        because latin-1 can always be used for any file (well, unless conversion
        from latin-1 to 'encoding' fails, which would be very bad).

        > I think the problem may be that CarbonVim is not linked with iconv, so
        > that it does not actually do any conversion, but I must also admit that
        > I still don't have a crystal clear understanding of the four above
        > variables...

        If iconv isn't present, you can see this in the output of ":version".
        And you can test it with the iconv() function.

        --
        hundred-and-one symptoms of being an internet addict:
        57. You begin to wonder how on earth your service provider is allowed to call
        200 hours per month "unlimited."

        /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
        /// Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim \\\
        \\\ Project leader for A-A-P -- http://www.a-a-p.org ///
        \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
      • Sébastien Pierre
        Hi! ... I tried: set termencoding=8bit-MacRoman set encoding=latin-1 set fileencodings=8bit-MacRoman and also tried to remove the termencoding. Both did no
        Message 3 of 14 , May 25, 2002
        • 0 Attachment
          Hi!

          Le samedi 25 mai 2002, à 01:27 PM, Bram Moolenaar a écrit :
          > When 'termencoding' is equal to 'encoding' or empty no conversion is
          > done.

          I tried:

          set termencoding=8bit-MacRoman
          set encoding=latin-1
          set fileencodings=8bit-MacRoman

          and also tried to remove the termencoding. Both did no solve the
          problem: I still have <8E> instead of <E9>.

          >> Removing the termencoding in my .vimrc did not change anything,
          >> removing
          >> latin-1 from the filencodings did nothing as well.
          >
          > Did you check the value of 'fileencoding' after loading a file?

          I can't remember nor find in the manual how to do that (echo just
          doesn't work with this kind of... variable?). I also tested with
          :fileencoding=latin-1 and :e --enc=latin 1 with the above configuration,
          and I still got the <8E>.

          I have checked with :version, and iconv has not been compiled in. This
          may be the reason why the encodings do not work properly?

          Thanks,

          -- Sébastien.

          --
          «Le soleil est Dieu»
          <http://www.type-z.org> -- Turner, avant de mourir
        • Bram Moolenaar
          ... Yes, without iconv no conversion is possible. -- Q: How to decide if I should clean my house or work on improving Vim? A: Depends on which has more bugs.
          Message 4 of 14 , May 25, 2002
          • 0 Attachment
            Sébastien Pierre wrote:

            > I can't remember nor find in the manual how to do that (echo just

            :set fileencoding

            > I have checked with :version, and iconv has not been compiled in. This
            > may be the reason why the encodings do not work properly?

            Yes, without iconv no conversion is possible.

            --
            Q: How to decide if I should clean my house or work on improving Vim?
            A: Depends on which has more bugs.

            /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
            /// Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim \\\
            \\\ Project leader for A-A-P -- http://www.a-a-p.org ///
            \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
          • Benji Fisher
            ... Should I add something like Enable font conversions with iconv to the list of bugs on my web page? Or is this something that I can add at compile time?
            Message 5 of 14 , May 25, 2002
            • 0 Attachment
              On Saturday, May 25, 2002, at 02:17 PM, Bram Moolenaar wrote:

              >
              > Sébastien Pierre wrote:
              >
              >> I have checked with :version, and iconv has not been compiled in. This
              >> may be the reason why the encodings do not work properly?
              >
              > Yes, without iconv no conversion is possible.

              Should I add something like "Enable font conversions with iconv" to
              the list of bugs on my web page? Or is this something that I can add at
              compile time?

              > --
              > Q: How to decide if I should clean my house or work on improving Vim?
              > A: Depends on which has more bugs.

              I think you should multiply the number of bugs by the number of
              people bothered by each bug. ;)

              --Benji Fisher
            • mc
              ... ... and divide by a function of the distance of the bothered people from the bugs in the house
              Message 6 of 14 , May 25, 2002
              • 0 Attachment
                >> Q: How to decide if I should clean my house or work on improving Vim?
                >> A: Depends on which has more bugs.
                >
                > I think you should multiply the number of bugs by the number of
                > people bothered by each bug. ;)


                ... and divide by a function of the distance of the bothered people from
                the bugs in the house
              • Sébastien Pierre
                ... For people who have Fink (like me), libiconv is available. The problem is that when you add -I/sw/include in CarbonVim makefile there are gcc duplicate
                Message 7 of 14 , May 26, 2002
                • 0 Attachment
                  Le dimanche 26 mai 2002, à 04:03 AM, Benji Fisher a écrit :

                  > On Saturday, May 25, 2002, at 02:17 PM, Bram Moolenaar wrote:
                  >
                  >>
                  >> Sébastien Pierre wrote:
                  >>
                  >>> I have checked with :version, and iconv has not been compiled in. This
                  >>> may be the reason why the encodings do not work properly?
                  >>
                  >> Yes, without iconv no conversion is possible.
                  >
                  > Should I add something like "Enable font conversions with iconv"
                  > to the list of bugs on my web page? Or is this something that I can
                  > add at compile time?

                  For people who have Fink (like me), libiconv is available. The problem
                  is that when you add -I/sw/include in CarbonVim makefile there are "gcc
                  duplicate declaration" errors. Iconv is a compilation option, so if you
                  know how to enable the makefile to look for additional libs in /sw/lib
                  and headers in /sw/include, this would be perfect :)

                  Thanks,

                  -- Sébastien.

                  --
                  «Big lies, big times / This love is not what we're about / It's
                  too late, and I'm too straight / It's time to blow this fire
                  out.»
                  <http://www.type-z.org> -- UNKLE, Psyence Fiction - Bloodstain
                • Bram Moolenaar
                  ... If you do have the iconv library, configure should find it automatically. However, I don t know if this works on the mac. You could try defining
                  Message 8 of 14 , May 26, 2002
                  • 0 Attachment
                    Benji Fisher wrote:

                    > > Sébastien Pierre wrote:
                    > >
                    > >> I have checked with :version, and iconv has not been compiled in. This
                    > >> may be the reason why the encodings do not work properly?
                    > >
                    > > Yes, without iconv no conversion is possible.
                    >
                    > Should I add something like "Enable font conversions with iconv" to
                    > the list of bugs on my web page? Or is this something that I can add at
                    > compile time?

                    If you do have the iconv library, configure should find it
                    automatically. However, I don't know if this works on the mac. You
                    could try defining USE_ICONV and see what happens. Perhaps define
                    HAVE_ICONV_H as well.

                    Another problem is that you might have the iconv library, but someone
                    who runs your compiled binary doesn't. The best way to solve this is by
                    dynamically loading the library. But currently this is only implemented
                    for MS-Windows. Would be a great help to have this on more systems!

                    > > --
                    > > Q: How to decide if I should clean my house or work on improving Vim?
                    > > A: Depends on which has more bugs.
                    >
                    > I think you should multiply the number of bugs by the number of
                    > people bothered by each bug. ;)

                    Then I need someone else to clean my house! :-)

                    --
                    hundred-and-one symptoms of being an internet addict:
                    60. As your car crashes through the guardrail on a mountain road, your first
                    instinct is to search for the "back" button.

                    /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
                    /// Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim \\\
                    \\\ Project leader for A-A-P -- http://www.a-a-p.org ///
                    \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
                  • MURAOKA Taro
                    ... MacOS X (Carbon) has a set of APIs which called Text Encoding Conversion Manager . It works as like as iconv library. So I have wrote iconv emulation
                    Message 9 of 14 , May 26, 2002
                    • 0 Attachment
                      > Another problem is that you might have the iconv library, but someone
                      > who runs your compiled binary doesn't. The best way to solve this is by
                      > dynamically loading the library. But currently this is only implemented
                      > for MS-Windows. Would be a great help to have this on more systems!

                      MacOS X (Carbon) has a set of APIs which called "Text Encoding
                      Conversion Manager". It works as like as iconv library. So I have
                      wrote iconv emulation layer for MacOS X using these APIs, and now I am
                      testing for Vim. This emulation layer is not so big as iconv library is,
                      so it could be included in vim.

                      Please try below patch and Makefile.
                      http://www.kaoriya.net/testdir/VimMacOSXIconvMakeSet.tar.bz2

                      In this layer, I add some aliases for Japansese encoding name. I'm
                      glad, if anyone add aliases for another languages (ex. Chinese, Korean)

                      Regards.
                      ----
                      MURAOKA Taro <koron@...>
                    Your message has been successfully submitted and would be delivered to recipients shortly.