Loading ...
Sorry, an error occurred while loading the content.

How to set up Outlook 2010 / Vim 7.3 to have OutlookVim with Unicode (UTF-8) support?

Expand Messages
  • aswna2010@...
    Hi all, hopefully, someone will be able to help me in the following issue. I have recently installed the OutlookVim Vim plugin
    Message 1 of 6 , Apr 6 11:02 AM
    • 0 Attachment
      Hi all,
      hopefully, someone will be able to help me in the following issue.

      I have recently installed the OutlookVim Vim plugin (https://github.com/vim-scripts/OutlookVim, version 8.0). I have MS Windows 7 Enterprise SP1 (64-bit OS) and Vim 7.3.46 (ftp://ftp.vim.org/pub/vim/pc/gvim73_46.exe).

      Basically, I would like to edit mails, which contain characters from the Hungarian alphabet (see below). Above setup does not seem to work out of the box for this purpose.

      I have set the following in Outlook 2010 trying to resolve the issue.
      Outlook -> File -> Options -> Advanced:
      - Automatically select encoding for outgoing messages (Enable)
      - Preferred encoding for outgoing messages: Unicode (UTF-8)
      - Automatically select encoding for outgoing vCards (Enable)
      - Preferred encoding for outgoing vCards: Unicode (UTF-8)
      - Allow UTF-8 support for the mailto: protocol (Enable)
      Using Central European (ISO) or Western European (Windows) give the same results.

      Current font settings in Outlook:
      Outlook -> File -> Options -> Mail -> Stationery and Fonts: Consolas in each case. (Have also tried Arial Unicode MS Regular, no luck.)

      In Vim (gVim) I have the guifont set as "Consolas:h11". In Vim I could not set the above Arial font (Arial\ Unicode\ MS\ Regular, Arial\ Unicode\ MS, etc did not work).

      Some characters, which cause trouble:
      U+0151 ő c5 91 LATIN SMALL LETTER O WITH DOUBLE ACUTE
      U+0171 ű c5 b1 LATIN SMALL LETTER U WITH DOUBLE ACUTE
      ...

      The issue (example): in Outlook I can enter and see the above characters correctly, but when I click on the Vim.Edit macro, which opens the text in Vim, above characters show up as 'o' and 'u'. :( When I save the buffer and gets back to Outlook the text is already got the wrong letters.

      I have also experimented with setting utf-8 encoding/fileencoding for the "mail" filetype, no luck. Also tried to hack the Vim.Edit macro, no success.

      Could someone share settings, which works for this or similar to this case?

      Thanks in advance!

      --
      --
      You received this message from the "vim_use" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_use" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • David Fishburn
      ... I am glad you are trying the multibyte support, I added it to OutlookVim, but have never had a chance to use it. There are a few things that are required
      Message 2 of 6 , Apr 7 5:02 PM
      • 0 Attachment

        On Sat, Apr 6, 2013 at 2:02 PM, <aswna2010@...> wrote:
        ...
         
        I have recently installed the OutlookVim Vim plugin (https://github.com/vim-scripts/OutlookVim, version 8.0). I have MS Windows 7 Enterprise SP1 (64-bit OS) and Vim 7.3.46 (ftp://ftp.vim.org/pub/vim/pc/gvim73_46.exe).

        Basically, I would like to edit mails, which contain characters from the Hungarian alphabet (see below). Above setup does not seem to work out of the box for this purpose.

        ...
         
        The issue (example): in Outlook I can enter and see the above characters correctly, but when I click on the Vim.Edit macro, which opens the text in Vim, above characters show up as 'o' and 'u'. :( When I save the buffer and gets back to Outlook the text is already got the wrong letters.

        I have also experimented with setting utf-8 encoding/fileencoding for the "mail" filetype, no luck. Also tried to hack the Vim.Edit macro, no success.

        I am glad you are trying the multibyte support, I added it to OutlookVim, but have never had a chance to use it.

        There are a few things that are required for Vim to support multibyte characters.
        NOTE: I do not use multibyte support in Vim, so if anyone else wants to chime in that would be terrific.

        The OutlookVim.bas file checks a few things in Vim:

        1.  Does your Vim support multibyte (test with :echo has("multi_byte") )?
        2.  Is a BOMB enabled (:set bomb, tested with :echo &bomb == 1)
        3.  Does the fileencodings support UTF8 ( tested with :echo match(&fileencodings, '\<ucs-bom\|utf\>')   )?


        If so, it will enable unicode format in the file which Outlook writes for Vim to edit.


        So, you can do this simple test.
        1.  Start Vim
        2.  :set bomb
        3.  :set filecodings=utf-8
        4.  :set encoding=utf-8
        5.  Make sure the guifont you are using supports the character sets you are displaying.
        6.  From Outlook, hit the Edit in Vim button again and see what happens.



        I have the following in my .vimrc when I want to play with multibyte:

        if has("gui_running")
            if has("multi_byte")
                set encoding=utf-8
                setglobal fileencoding=utf-8
                set bomb
                set termencoding=iso-8859-15
                set fileencodings=iso-8859-15,iso-8859-3,ucs-bom,utf-8,utf-16

                " Enable Unicode (for Chinese and so on)
                " Downloaded from here: http://sourceforge.net/projects/dejavu/
                if filereadable(expand('$SystemRoot').'/fonts/DejaVuSansMono.ttf')
                    " If running on drastically different resolution monitors
                    " this will set the font based which screen you are
                    " currently on.  This assumes the monitor above you has
                    " a higher resolution.
                    let fontsize = (getwinposx()<0 ? '11' : '11')
                    " For normal 11 point font
                    "   set guifont=Bitstream_Vera_Sans_Mono:h11:cANSI
                    " For bold 11 point font
                    " echomsg 'fontsize:'.fontsize.'  posx:'.getwinposx()
                    exec 'set guifont=DejaVu_Sans_Mono:h'.fontsize.':cANSI'
                endif
            else
                echoerr "Sorry, this version of (g)vim was not compiled with +multi_byte"
            endif
        else
            echoerr "console Vim will not display UTF-8 properly"
        endif
         

        HTH,
        David

        --
        --
        You received this message from the "vim_use" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
         
        ---
        You received this message because you are subscribed to the Google Groups "vim_use" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
         
         
      • aswna2010@...
        ... 1. Yes (1). 2. No (nobomb). 3. Yes (0). Last time, as I enabled some of the MsgBox messages in the macro, I could see, it set the unicode variable to True.
        Message 3 of 6 , Apr 8 1:59 AM
        • 0 Attachment
          > 1.  Does your Vim support multibyte (test with :echo has("multi_byte") )?
          > 2.  Is a BOMB enabled (:set bomb, tested with :echo &bomb == 1)
          >
          > 3.  Does the fileencodings support UTF8 ( tested with :echo match(&fileencodings, '\<ucs-bom\|utf\>')   )?
          1. Yes (1).
          2. No (nobomb).
          3. Yes (0).

          Last time, as I enabled some of the MsgBox messages in the macro, I could see, it set the unicode variable to True.
          Note: a bit strange for me, that after the macro uses the same variable and sets it to False, and then "Check[s] if there are any unicode characters in the body". Here the macro did not set unicode variable back to True (despite of above characters present in the mail body). I played with keeping the unicode variable to True, in fact the file written out and read by Vim seemed to be Unicode (UTF-16 LE: <fffe>), but Vim could not display it correctly for me, with my settings.

          > So, you can do this simple test.
          > 1.  Start Vim
          > 2.  :set bomb
          > 3.  :set filecodings=utf-8
          > 4.  :set encoding=utf-8

          For the record, in 3 it is fileencodings.
          Setting bomb has no effect on the new buffer, so I have set the above in the .vim/after/ftplugin/mail.vim.

          > 5.  Make sure the guifont you are using supports the character sets you are displaying.
          I am not 100% sure about this. I can tell, that it is possible to enter and see the same characters in Vim directly, but I am not sure if these are really Unicode characters.

          > 6.  From Outlook, hit the Edit in Vim button again and see what happens.
          The result is the same.

          > I have the following in my .vimrc when I want to play with multibyte:
          > ...
          I have downloaded (and could set DejaVu Sans Mono), set the same config, but it did not help. :(

          > HTH,
          > David
          Thanks for the suggestions!

          Any other tips?

          --
          --
          You received this message from the "vim_use" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_use" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • David Fishburn
          ... Let s start with verifying the settings in Vim are compatible with the characters we are trying to get from Outlook. You indicated above, that you can
          Message 4 of 6 , Apr 8 5:33 AM
          • 0 Attachment



            On Mon, Apr 8, 2013 at 4:59 AM, <aswna2010@...> wrote:
            > 1.  Does your Vim support multibyte (test with :echo has("multi_byte") )?
            > 2.  Is a BOMB enabled (:set bomb, tested with :echo &bomb == 1)
            >
            > 3.  Does the fileencodings support UTF8 ( tested with :echo match(&fileencodings, '\<ucs-bom\|utf\>')   )?
            1. Yes (1).
            2. No (nobomb).
            3. Yes (0).

            Last time, as I enabled some of the MsgBox messages in the macro, I could see, it set the unicode variable to True.
            Note: a bit strange for me, that after the macro uses the same variable and sets  it to False, and then "Check[s] if there are any unicode characters in the body". Here the macro did not set unicode variable back to True (despite of above characters present in the mail body). I played with keeping the unicode variable to True, in fact the file written out and read by Vim seemed to be Unicode (UTF-16 LE: <fffe>), but Vim could not display it correctly for me, with my settings.

            > So, you can do this simple test.
            > 1.  Start Vim
            > 2.  :set bomb
            > 3.  :set filecodings=utf-8
            > 4.  :set encoding=utf-8

            For the record, in 3 it is fileencodings.
            Setting bomb has no effect on the new buffer, so I have set the above in the .vim/after/ftplugin/mail.vim.

            > 5.  Make sure the guifont you are using supports the character sets you are displaying.
            I am not 100% sure about this. I can tell, that it is possible to enter and see the same characters in Vim directly, but I am not sure if these are really Unicode characters.

            > 6.  From Outlook, hit the Edit in Vim button again and see what happens.
            The result is the same.

            > I have the following in my .vimrc when I want to play with multibyte:
            > ...
            I have downloaded (and could set DejaVu Sans Mono), set the same config, but it did not help. :(



            Let's start with verifying the settings in Vim are compatible with the characters we are trying to get from Outlook.  You indicated above, that you can enter those characters in your Vim by yourself.

            Lets try this then.
            Start Vim.
            Set the required options (encodings, fileencodings).
            Copy and paste the body of the Outlook message into Vim.

            When you do it this way, how do they show up?

            Assuming all was good, save the file.

            In the OutlookVim.bas file (as you stated above), force the unicode to be True, and Edit in Vim.

            What I am interested in is the differences in the file between the one you copy and pasted, and the one that Outlook saved.  It might even require hex mode to have a look to see what might be different.

            David 

            --
            --
            You received this message from the "vim_use" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
             
            ---
            You received this message because you are subscribed to the Google Groups "vim_use" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
             
             
          • aswna2010@...
            Hi David ... Yes, it was successful. I have sourced the following .vim after ftplugin mail.vim: set bomb set encoding=utf-8 set fileencoding=utf-8 set
            Message 5 of 6 , Apr 8 7:31 AM
            • 0 Attachment
              Hi David
              > Assuming all was good, save the file.
              Yes, it was successful.

              I have sourced the following .vim\after\ftplugin\mail.vim:
              set bomb
              set encoding=utf-8
              set fileencoding=utf-8
              set termencoding=utf-8

              The hexdump results of the two-character long mail body of "őű".
              1. Copy-paste (from Outlook to Vim).
              0000000 bbef c5bf c591 0db1 000a
              0000009

              2. Using the macro as it is.
              0000000 bbef 6fbf 0a75
              0000006

              3. Modified the macro to force unicode (utf-16).
              0000000 bbef efbf bebf 84e5 e781 8184 a8e0 0d80
              0000010 000a
              0000011

              4. Modified the macro the force unicode (utf-8).
              0000000 bbef 3fbf 513f 7101 0a01 0a00
              000000c

              Here I called "force unicode": commented line 201 in the macro, where unicode variable is set to False.

              Thanks!

              --
              --
              You received this message from the "vim_use" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_use" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
            • aswna2010@...
              As I mentioned in the other thread [Re: UFT-8/16 help needed (How to set up Outlook 2010 / Vim 7.3 to have OutlookVim with Unicode (UTF-8) support?)], I have
              Message 6 of 6 , Apr 9 11:50 PM
              • 0 Attachment
                As I mentioned in the other thread [Re: UFT-8/16 help needed (How to set up Outlook 2010 / Vim 7.3 to have OutlookVim with Unicode (UTF-8) support?)], I have experimented some more with the plugin.

                The solution what works for me can be seen with the help of the following diff.

                diff -b original_OutlookVim.bas working_OutlookVim.bas
                201c201
                < unicode = False
                ---
                > 'unicode = False
                289c289
                < vimKeys = vimKeys & "utf-16"
                ---
                > vimKeys = vimKeys & "utf-16le"

                I am not sure how would you fix this, if at all.

                Now, it really produces the expected characters, the hexdump also confirms this.
                0000000 feff 0151 0171 000a
                0000008

                Thanks for your help,
                András

                --
                --
                You received this message from the "vim_use" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_use" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              Your message has been successfully submitted and would be delivered to recipients shortly.