Loading ...
Sorry, an error occurred while loading the content.

translate unicode

Expand Messages
  • bill lam
    After converting m$ word doc to tex, there are some embedding strings such as, { selectlanguage{english} sffamily bfseries To: [6052?][8C50?]} Those 4 hex
    Message 1 of 14 , Nov 1, 2008
    • 0 Attachment
      After converting m$ word doc to tex, there are some embedding strings
      such as,
      {\selectlanguage{english}\sffamily\bfseries
      To: [6052?][8C50?]}

      Those 4 hex digit inside square brackets are chinese characters, eg,
      that lines should be
      {\selectlanguage{english}\sffamily\bfseries
      To: 恒豐}

      instead of manually typing the strings again using ctrl-v u, will that
      be any shortcuts?

      PS. some additional katex cjk macros will still be needed to
      constitute a correct chinese tex file but that will be another issue
      not related to vim.

      --
      regards,
      ====================================================
      GPG key 1024D/4434BAB3 2008-08-24
      gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_use" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • Tony Mechelynck
      ... Well, you can use the :s[ubstitute] command to convert the codes for you. Let s see if I remember how I did it once, a year or two ago. First, of course,
      Message 2 of 14 , Nov 1, 2008
      • 0 Attachment
        On 01/11/08 15:33, bill lam wrote:
        > After converting m$ word doc to tex, there are some embedding strings
        > such as,
        > {\selectlanguage{english}\sffamily\bfseries
        > To: [6052?][8C50?]}
        >
        > Those 4 hex digit inside square brackets are chinese characters, eg,
        > that lines should be
        > {\selectlanguage{english}\sffamily\bfseries
        > To: 恒豐}
        >
        > instead of manually typing the strings again using ctrl-v u, will that
        > be any shortcuts?
        >
        > PS. some additional katex cjk macros will still be needed to
        > constitute a correct chinese tex file but that will be another issue
        > not related to vim.
        >

        Well, you can use the :s[ubstitute] command to convert the codes for
        you. Let's see if I remember how I did it once, a year or two ago.

        First, of course, you need a Vim compiled with +multi_byte running with
        'encoding' set to utf-8. What follows will assume that you already use
        that. If the editfile's disk representation is something else (such as
        GB18030) that's no problem as long as 'fileencoding' is set correctly,
        either by 'fileencodings' or by ++enc. But you know that, I'm sure.

        :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g

        I didn't test it today, so be ready to use undo if it doesn't work.
        Also, the above assumes that your [xxxx?] codes always include exactly 4
        hex digits, which implies no codepoint above U+FFFF (it should work
        below U+1000 if leading zeros are included). UTF-16 surrogates are also
        not handled.


        Best regards,
        Tony.
        --
        hundred-and-one symptoms of being an internet addict:
        239. You think "surfing" is something you do on dry land.

        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_use" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • bill lam
        ... Thanks. This works. Those files were edited using m$ ime that did not support codepoint above U+10000. A related question. I have a number of such files
        Message 3 of 14 , Nov 1, 2008
        • 0 Attachment
          On Sat, 01 Nov 2008, Tony Mechelynck wrote:
          >
          > :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g

          Thanks. This works. Those files were edited using m$ ime that did not
          support codepoint above U+10000.

          A related question. I have a number of such files that need fixes.
          How do I grep and replace them all under linux.

          --
          regards,
          ====================================================
          GPG key 1024D/4434BAB3 2008-08-24
          gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_use" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Tony Mechelynck
          ... from pattern and to expression same as above any number of files (with wildcards if desired) in the :args command Best regards, Tony. -- Jones s First
          Message 4 of 14 , Nov 1, 2008
          • 0 Attachment
            On 02/11/08 04:37, bill lam wrote:
            > On Sat, 01 Nov 2008, Tony Mechelynck wrote:
            >> :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g
            >
            > Thanks. This works. Those files were edited using m$ ime that did not
            > support codepoint above U+10000.
            >
            > A related question. I have a number of such files that need fixes.
            > How do I grep and replace them all under linux.
            >

            :args file1 file2 *.tex foo???bar.baz
            :argdo %s/...from.../...to.../ge

            "from" pattern and "to" expression same as above
            any number of files (with wildcards if desired) in the :args command


            Best regards,
            Tony.
            --
            Jones's First Law:
            Anyone who makes a significant contribution to any field of
            endeavor, and stays in that field long enough, becomes an obstruction
            to its progress -- in direct proportion to the importance of their
            original contribution.

            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_use" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          • bill lam
            ... Thanks! I found that it also needs autowrite and argdo!. -- regards, ==================================================== GPG key 1024D/4434BAB3
            Message 5 of 14 , Nov 2, 2008
            • 0 Attachment
              On Sun, 02 Nov 2008, Tony Mechelynck wrote:

              >
              > On 02/11/08 04:37, bill lam wrote:
              > > On Sat, 01 Nov 2008, Tony Mechelynck wrote:
              > >> :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g
              > >
              > > Thanks. This works. Those files were edited using m$ ime that did not
              > > support codepoint above U+10000.
              > >
              > > A related question. I have a number of such files that need fixes.
              > > How do I grep and replace them all under linux.
              > >
              >
              > :args file1 file2 *.tex foo???bar.baz
              > :argdo %s/...from.../...to.../ge
              >
              > "from" pattern and "to" expression same as above
              > any number of files (with wildcards if desired) in the :args command
              >

              Thanks! I found that it also needs autowrite and argdo!.

              --
              regards,
              ====================================================
              GPG key 1024D/4434BAB3 2008-08-24
              gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_use" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            • Tony Mechelynck
              ... ah, yes, you may add |update at the end of the argdo line to write the file (if changed) after the substitute, and :new before :argdo to avoid problems if
              Message 6 of 14 , Nov 2, 2008
              • 0 Attachment
                On 02/11/08 08:04, bill lam wrote:
                > On Sun, 02 Nov 2008, Tony Mechelynck wrote:
                >
                >> On 02/11/08 04:37, bill lam wrote:
                >>> On Sat, 01 Nov 2008, Tony Mechelynck wrote:
                >>>> :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g
                >>> Thanks. This works. Those files were edited using m$ ime that did not
                >>> support codepoint above U+10000.
                >>>
                >>> A related question. I have a number of such files that need fixes.
                >>> How do I grep and replace them all under linux.
                >>>
                >> :args file1 file2 *.tex foo???bar.baz
                >> :argdo %s/...from.../...to.../ge
                >>
                >> "from" pattern and "to" expression same as above
                >> any number of files (with wildcards if desired) in the :args command
                >>
                >
                > Thanks! I found that it also needs autowrite and argdo!.
                >

                ah, yes, you may add |update at the end of the argdo line to write the
                file (if changed) after the substitute, and :new before :argdo to avoid
                problems if the current file is 'modified' (with :argdo! if the current
                file is 'modified' the changes will be lost), or, like me (and this is
                why I forgot) you may set 'autowriteall' in your vimrc.

                Best regards,
                Tony.
                --
                hundred-and-one symptoms of being an internet addict:
                243. You unsuccessfully try to download a pizza from www.dominos.com.

                --~--~---------~--~----~------------~-------~--~----~
                You received this message from the "vim_use" maillist.
                For more information, visit http://www.vim.org/maillist.php
                -~----------~----~----~----~------~----~------~--~---
              • bill lam
                ... Still one more question, when running the argdo, the output was directed to a pager that need manual response to each - more - prompt, (I have several
                Message 7 of 14 , Nov 2, 2008
                • 0 Attachment
                  On Sun, 02 Nov 2008, Tony Mechelynck wrote:
                  > ah, yes, you may add |update at the end of the argdo line to write the
                  > file (if changed) after the substitute, and :new before :argdo to avoid
                  > problems if the current file is 'modified' (with :argdo! if the current
                  > file is 'modified' the changes will be lost), or, like me (and this is
                  > why I forgot) you may set 'autowriteall' in your vimrc.

                  Still one more question, when running the argdo, the output was
                  directed to a pager that need manual response to each '- more -'
                  prompt, (I have several hundreds of files), how can this be avoided?

                  --
                  regards,
                  ====================================================
                  GPG key 1024D/4434BAB3 2008-08-24
                  gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

                  --~--~---------~--~----~------------~-------~--~----~
                  You received this message from the "vim_use" maillist.
                  For more information, visit http://www.vim.org/maillist.php
                  -~----------~----~----~----~------~----~------~--~---
                • Ben Schmidt
                  ... Ben. --~--~---------~--~----~------------~-------~--~----~ You received this message from the vim_use maillist. For more information, visit
                  Message 8 of 14 , Nov 2, 2008
                  • 0 Attachment
                    bill lam wrote:
                    > On Sun, 02 Nov 2008, Tony Mechelynck wrote:
                    >> ah, yes, you may add |update at the end of the argdo line to write the
                    >> file (if changed) after the substitute, and :new before :argdo to avoid
                    >> problems if the current file is 'modified' (with :argdo! if the current
                    >> file is 'modified' the changes will be lost), or, like me (and this is
                    >> why I forgot) you may set 'autowriteall' in your vimrc.
                    >
                    > Still one more question, when running the argdo, the output was
                    > directed to a pager that need manual response to each '- more -'
                    > prompt, (I have several hundreds of files), how can this be avoided?

                    :set nomore

                    :help 'more'

                    Ben.




                    --~--~---------~--~----~------------~-------~--~----~
                    You received this message from the "vim_use" maillist.
                    For more information, visit http://www.vim.org/maillist.php
                    -~----------~----~----~----~------~----~------~--~---
                  • bill lam
                    ... Thank you Ben. I notice sometimes it need quote when :help, sometimes not? eg. :h cb and :h cb give different help pages, what is the rule for that
                    Message 9 of 14 , Nov 2, 2008
                    • 0 Attachment
                      On Mon, 03 Nov 2008, Ben Schmidt wrote:
                      > :set nomore
                      >
                      > :help 'more'

                      Thank you Ben.

                      I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                      and :h 'cb' give different help pages, what is the rule for that
                      quote?

                      --
                      regards,
                      ====================================================
                      GPG key 1024D/4434BAB3 2008-08-24
                      gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

                      --~--~---------~--~----~------------~-------~--~----~
                      You received this message from the "vim_use" maillist.
                      For more information, visit http://www.vim.org/maillist.php
                      -~----------~----~----~----~------~----~------~--~---
                    • Tony Mechelynck
                      ... Quotes are for an option: :help cb tells you what you can :set cb to. Nothing is for Normal-mode commands i_ for Insert, c_ for Command-line, v_ for
                      Message 10 of 14 , Nov 2, 2008
                      • 0 Attachment
                        On 03/11/08 07:39, bill lam wrote:
                        > On Mon, 03 Nov 2008, Ben Schmidt wrote:
                        >> :set nomore
                        >>
                        >> :help 'more'
                        >
                        > Thank you Ben.
                        >
                        > I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                        > and :h 'cb' give different help pages, what is the rule for that
                        > quote?
                        >

                        Quotes are for an option: ":help 'cb'" tells you what you can ":set cb" to.

                        Nothing is for Normal-mode commands

                        i_ for Insert, c_ for Command-line, v_ for Visual, : for Ex-commands, -
                        for command-line switches, etc.

                        See ":help help-context" on the very first page of help (where you land
                        when you hit F1 in Vim).


                        Best regards,
                        Tony.
                        --
                        "What is wanted is not the will to believe, but the will to find out,
                        which is the exact opposite."
                        -- Bertrand Russell, "Skeptical_Essays", 1928

                        --~--~---------~--~----~------------~-------~--~----~
                        You received this message from the "vim_use" maillist.
                        For more information, visit http://www.vim.org/maillist.php
                        -~----------~----~----~----~------~----~------~--~---
                      • Ben Schmidt
                        ... The quote is for options. Just do ... all by itself and look at the Get specific help section, which explains the help tag conventions. It also helps to
                        Message 11 of 14 , Nov 2, 2008
                        • 0 Attachment
                          bill lam wrote:
                          > On Mon, 03 Nov 2008, Ben Schmidt wrote:
                          >> :set nomore
                          >>
                          >> :help 'more'
                          >
                          > Thank you Ben.
                          >
                          > I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                          > and :h 'cb' give different help pages, what is the rule for that
                          > quote?

                          The quote is for options. Just do

                          :help

                          all by itself and look at the 'Get specific help' section, which
                          explains the help tag conventions. It also helps to know that control
                          keys are shown like CTRL-A in help tags, and special keys like <BS>, and
                          tags involving such keys have underscores to separate the parts, e.g.
                          CTRL-W_<Up>.

                          :help <>

                          is also useful to look at.

                          Cheers,

                          Ben.




                          --~--~---------~--~----~------------~-------~--~----~
                          You received this message from the "vim_use" maillist.
                          For more information, visit http://www.vim.org/maillist.php
                          -~----------~----~----~----~------~----~------~--~---
                        • Ben Schmidt
                          ... And I should also mention ... If you scroll down a bit from there, you ll also find info on the useful ... Ben.
                          Message 12 of 14 , Nov 2, 2008
                          • 0 Attachment
                            Ben Schmidt wrote:
                            > bill lam wrote:
                            >> On Mon, 03 Nov 2008, Ben Schmidt wrote:
                            >>> :set nomore
                            >>>
                            >>> :help 'more'
                            >> Thank you Ben.
                            >>
                            >> I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                            >> and :h 'cb' give different help pages, what is the rule for that
                            >> quote?
                            >
                            > The quote is for options. Just do
                            >
                            > :help
                            >
                            > all by itself and look at the 'Get specific help' section, which
                            > explains the help tag conventions. It also helps to know that control
                            > keys are shown like CTRL-A in help tags, and special keys like <BS>, and
                            > tags involving such keys have underscores to separate the parts, e.g.
                            > CTRL-W_<Up>.
                            >
                            > :help <>
                            >
                            > is also useful to look at.

                            And I should also mention

                            :help :help

                            If you scroll down a bit from there, you'll also find info on the useful
                            :helpgrep and other such tips for using the help.

                            Ben.




                            --~--~---------~--~----~------------~-------~--~----~
                            You received this message from the "vim_use" maillist.
                            For more information, visit http://www.vim.org/maillist.php
                            -~----------~----~----~----~------~----~------~--~---
                          • bill lam
                            ... Thank you Tony and Ben. I once had read(browse) that :help help but can not notice the quotes are for an option. -- regards,
                            Message 13 of 14 , Nov 2, 2008
                            • 0 Attachment
                              On Mon, 03 Nov 2008, Tony Mechelynck wrote:

                              >
                              > On 03/11/08 07:39, bill lam wrote:
                              > > On Mon, 03 Nov 2008, Ben Schmidt wrote:
                              > >> :set nomore
                              > >>
                              > >> :help 'more'
                              > >
                              > > Thank you Ben.
                              > >
                              > > I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                              > > and :h 'cb' give different help pages, what is the rule for that
                              > > quote?
                              > >
                              >
                              > Quotes are for an option: ":help 'cb'" tells you what you can ":set cb" to.
                              >
                              > Nothing is for Normal-mode commands
                              >
                              > i_ for Insert, c_ for Command-line, v_ for Visual, : for Ex-commands, -
                              > for command-line switches, etc.
                              >
                              > See ":help help-context" on the very first page of help (where you land
                              > when you hit F1 in Vim).
                              >

                              Thank you Tony and Ben.

                              I once had read(browse) that :help help but can not notice the quotes
                              are for an option.

                              --
                              regards,
                              ====================================================
                              GPG key 1024D/4434BAB3 2008-08-24
                              gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

                              --~--~---------~--~----~------------~-------~--~----~
                              You received this message from the "vim_use" maillist.
                              For more information, visit http://www.vim.org/maillist.php
                              -~----------~----~----~----~------~----~------~--~---
                            • Tony Mechelynck
                              ... at the bottom of the list: Option :help textwidth Best regards, Tony. -- hundred-and-one symptoms of being an internet addict: 246. You use up
                              Message 14 of 14 , Nov 3, 2008
                              • 0 Attachment
                                On 03/11/08 08:41, bill lam wrote:
                                > On Mon, 03 Nov 2008, Tony Mechelynck wrote:
                                >
                                >> On 03/11/08 07:39, bill lam wrote:
                                >>> On Mon, 03 Nov 2008, Ben Schmidt wrote:
                                >>>> :set nomore
                                >>>>
                                >>>> :help 'more'
                                >>> Thank you Ben.
                                >>>
                                >>> I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                                >>> and :h 'cb' give different help pages, what is the rule for that
                                >>> quote?
                                >>>
                                >> Quotes are for an option: ":help 'cb'" tells you what you can ":set cb" to.
                                >>
                                >> Nothing is for Normal-mode commands
                                >>
                                >> i_ for Insert, c_ for Command-line, v_ for Visual, : for Ex-commands, -
                                >> for command-line switches, etc.
                                >>
                                >> See ":help help-context" on the very first page of help (where you land
                                >> when you hit F1 in Vim).
                                >>
                                >
                                > Thank you Tony and Ben.
                                >
                                > I once had read(browse) that :help help but can not notice the quotes
                                > are for an option.
                                >

                                at the bottom of the list:

                                Option ' :help 'textwidth'


                                Best regards,
                                Tony.
                                --
                                hundred-and-one symptoms of being an internet addict:
                                246. You use up your free 100 hours in less than a week.

                                --~--~---------~--~----~------------~-------~--~----~
                                You received this message from the "vim_use" maillist.
                                For more information, visit http://www.vim.org/maillist.php
                                -~----------~----~----~----~------~----~------~--~---
                              Your message has been successfully submitted and would be delivered to recipients shortly.