Loading ...
Sorry, an error occurred while loading the content.

Re: translate unicode

Expand Messages
  • Tony Mechelynck
    ... Well, you can use the :s[ubstitute] command to convert the codes for you. Let s see if I remember how I did it once, a year or two ago. First, of course,
    Message 1 of 14 , Nov 1, 2008
    • 0 Attachment
      On 01/11/08 15:33, bill lam wrote:
      > After converting m$ word doc to tex, there are some embedding strings
      > such as,
      > {\selectlanguage{english}\sffamily\bfseries
      > To: [6052?][8C50?]}
      >
      > Those 4 hex digit inside square brackets are chinese characters, eg,
      > that lines should be
      > {\selectlanguage{english}\sffamily\bfseries
      > To: 恒豐}
      >
      > instead of manually typing the strings again using ctrl-v u, will that
      > be any shortcuts?
      >
      > PS. some additional katex cjk macros will still be needed to
      > constitute a correct chinese tex file but that will be another issue
      > not related to vim.
      >

      Well, you can use the :s[ubstitute] command to convert the codes for
      you. Let's see if I remember how I did it once, a year or two ago.

      First, of course, you need a Vim compiled with +multi_byte running with
      'encoding' set to utf-8. What follows will assume that you already use
      that. If the editfile's disk representation is something else (such as
      GB18030) that's no problem as long as 'fileencoding' is set correctly,
      either by 'fileencodings' or by ++enc. But you know that, I'm sure.

      :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g

      I didn't test it today, so be ready to use undo if it doesn't work.
      Also, the above assumes that your [xxxx?] codes always include exactly 4
      hex digits, which implies no codepoint above U+FFFF (it should work
      below U+1000 if leading zeros are included). UTF-16 surrogates are also
      not handled.


      Best regards,
      Tony.
      --
      hundred-and-one symptoms of being an internet addict:
      239. You think "surfing" is something you do on dry land.

      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_use" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • bill lam
      ... Thanks. This works. Those files were edited using m$ ime that did not support codepoint above U+10000. A related question. I have a number of such files
      Message 2 of 14 , Nov 1, 2008
      • 0 Attachment
        On Sat, 01 Nov 2008, Tony Mechelynck wrote:
        >
        > :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g

        Thanks. This works. Those files were edited using m$ ime that did not
        support codepoint above U+10000.

        A related question. I have a number of such files that need fixes.
        How do I grep and replace them all under linux.

        --
        regards,
        ====================================================
        GPG key 1024D/4434BAB3 2008-08-24
        gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_use" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • Tony Mechelynck
        ... from pattern and to expression same as above any number of files (with wildcards if desired) in the :args command Best regards, Tony. -- Jones s First
        Message 3 of 14 , Nov 1, 2008
        • 0 Attachment
          On 02/11/08 04:37, bill lam wrote:
          > On Sat, 01 Nov 2008, Tony Mechelynck wrote:
          >> :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g
          >
          > Thanks. This works. Those files were edited using m$ ime that did not
          > support codepoint above U+10000.
          >
          > A related question. I have a number of such files that need fixes.
          > How do I grep and replace them all under linux.
          >

          :args file1 file2 *.tex foo???bar.baz
          :argdo %s/...from.../...to.../ge

          "from" pattern and "to" expression same as above
          any number of files (with wildcards if desired) in the :args command


          Best regards,
          Tony.
          --
          Jones's First Law:
          Anyone who makes a significant contribution to any field of
          endeavor, and stays in that field long enough, becomes an obstruction
          to its progress -- in direct proportion to the importance of their
          original contribution.

          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_use" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • bill lam
          ... Thanks! I found that it also needs autowrite and argdo!. -- regards, ==================================================== GPG key 1024D/4434BAB3
          Message 4 of 14 , Nov 2, 2008
          • 0 Attachment
            On Sun, 02 Nov 2008, Tony Mechelynck wrote:

            >
            > On 02/11/08 04:37, bill lam wrote:
            > > On Sat, 01 Nov 2008, Tony Mechelynck wrote:
            > >> :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g
            > >
            > > Thanks. This works. Those files were edited using m$ ime that did not
            > > support codepoint above U+10000.
            > >
            > > A related question. I have a number of such files that need fixes.
            > > How do I grep and replace them all under linux.
            > >
            >
            > :args file1 file2 *.tex foo???bar.baz
            > :argdo %s/...from.../...to.../ge
            >
            > "from" pattern and "to" expression same as above
            > any number of files (with wildcards if desired) in the :args command
            >

            Thanks! I found that it also needs autowrite and argdo!.

            --
            regards,
            ====================================================
            GPG key 1024D/4434BAB3 2008-08-24
            gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_use" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          • Tony Mechelynck
            ... ah, yes, you may add |update at the end of the argdo line to write the file (if changed) after the substitute, and :new before :argdo to avoid problems if
            Message 5 of 14 , Nov 2, 2008
            • 0 Attachment
              On 02/11/08 08:04, bill lam wrote:
              > On Sun, 02 Nov 2008, Tony Mechelynck wrote:
              >
              >> On 02/11/08 04:37, bill lam wrote:
              >>> On Sat, 01 Nov 2008, Tony Mechelynck wrote:
              >>>> :%s/\[\(\x\x\x\x\)?]/\=eval('"\u' . submatch(1) . '"')/g
              >>> Thanks. This works. Those files were edited using m$ ime that did not
              >>> support codepoint above U+10000.
              >>>
              >>> A related question. I have a number of such files that need fixes.
              >>> How do I grep and replace them all under linux.
              >>>
              >> :args file1 file2 *.tex foo???bar.baz
              >> :argdo %s/...from.../...to.../ge
              >>
              >> "from" pattern and "to" expression same as above
              >> any number of files (with wildcards if desired) in the :args command
              >>
              >
              > Thanks! I found that it also needs autowrite and argdo!.
              >

              ah, yes, you may add |update at the end of the argdo line to write the
              file (if changed) after the substitute, and :new before :argdo to avoid
              problems if the current file is 'modified' (with :argdo! if the current
              file is 'modified' the changes will be lost), or, like me (and this is
              why I forgot) you may set 'autowriteall' in your vimrc.

              Best regards,
              Tony.
              --
              hundred-and-one symptoms of being an internet addict:
              243. You unsuccessfully try to download a pizza from www.dominos.com.

              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_use" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            • bill lam
              ... Still one more question, when running the argdo, the output was directed to a pager that need manual response to each - more - prompt, (I have several
              Message 6 of 14 , Nov 2, 2008
              • 0 Attachment
                On Sun, 02 Nov 2008, Tony Mechelynck wrote:
                > ah, yes, you may add |update at the end of the argdo line to write the
                > file (if changed) after the substitute, and :new before :argdo to avoid
                > problems if the current file is 'modified' (with :argdo! if the current
                > file is 'modified' the changes will be lost), or, like me (and this is
                > why I forgot) you may set 'autowriteall' in your vimrc.

                Still one more question, when running the argdo, the output was
                directed to a pager that need manual response to each '- more -'
                prompt, (I have several hundreds of files), how can this be avoided?

                --
                regards,
                ====================================================
                GPG key 1024D/4434BAB3 2008-08-24
                gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

                --~--~---------~--~----~------------~-------~--~----~
                You received this message from the "vim_use" maillist.
                For more information, visit http://www.vim.org/maillist.php
                -~----------~----~----~----~------~----~------~--~---
              • Ben Schmidt
                ... Ben. --~--~---------~--~----~------------~-------~--~----~ You received this message from the vim_use maillist. For more information, visit
                Message 7 of 14 , Nov 2, 2008
                • 0 Attachment
                  bill lam wrote:
                  > On Sun, 02 Nov 2008, Tony Mechelynck wrote:
                  >> ah, yes, you may add |update at the end of the argdo line to write the
                  >> file (if changed) after the substitute, and :new before :argdo to avoid
                  >> problems if the current file is 'modified' (with :argdo! if the current
                  >> file is 'modified' the changes will be lost), or, like me (and this is
                  >> why I forgot) you may set 'autowriteall' in your vimrc.
                  >
                  > Still one more question, when running the argdo, the output was
                  > directed to a pager that need manual response to each '- more -'
                  > prompt, (I have several hundreds of files), how can this be avoided?

                  :set nomore

                  :help 'more'

                  Ben.




                  --~--~---------~--~----~------------~-------~--~----~
                  You received this message from the "vim_use" maillist.
                  For more information, visit http://www.vim.org/maillist.php
                  -~----------~----~----~----~------~----~------~--~---
                • bill lam
                  ... Thank you Ben. I notice sometimes it need quote when :help, sometimes not? eg. :h cb and :h cb give different help pages, what is the rule for that
                  Message 8 of 14 , Nov 2, 2008
                  • 0 Attachment
                    On Mon, 03 Nov 2008, Ben Schmidt wrote:
                    > :set nomore
                    >
                    > :help 'more'

                    Thank you Ben.

                    I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                    and :h 'cb' give different help pages, what is the rule for that
                    quote?

                    --
                    regards,
                    ====================================================
                    GPG key 1024D/4434BAB3 2008-08-24
                    gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

                    --~--~---------~--~----~------------~-------~--~----~
                    You received this message from the "vim_use" maillist.
                    For more information, visit http://www.vim.org/maillist.php
                    -~----------~----~----~----~------~----~------~--~---
                  • Tony Mechelynck
                    ... Quotes are for an option: :help cb tells you what you can :set cb to. Nothing is for Normal-mode commands i_ for Insert, c_ for Command-line, v_ for
                    Message 9 of 14 , Nov 2, 2008
                    • 0 Attachment
                      On 03/11/08 07:39, bill lam wrote:
                      > On Mon, 03 Nov 2008, Ben Schmidt wrote:
                      >> :set nomore
                      >>
                      >> :help 'more'
                      >
                      > Thank you Ben.
                      >
                      > I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                      > and :h 'cb' give different help pages, what is the rule for that
                      > quote?
                      >

                      Quotes are for an option: ":help 'cb'" tells you what you can ":set cb" to.

                      Nothing is for Normal-mode commands

                      i_ for Insert, c_ for Command-line, v_ for Visual, : for Ex-commands, -
                      for command-line switches, etc.

                      See ":help help-context" on the very first page of help (where you land
                      when you hit F1 in Vim).


                      Best regards,
                      Tony.
                      --
                      "What is wanted is not the will to believe, but the will to find out,
                      which is the exact opposite."
                      -- Bertrand Russell, "Skeptical_Essays", 1928

                      --~--~---------~--~----~------------~-------~--~----~
                      You received this message from the "vim_use" maillist.
                      For more information, visit http://www.vim.org/maillist.php
                      -~----------~----~----~----~------~----~------~--~---
                    • Ben Schmidt
                      ... The quote is for options. Just do ... all by itself and look at the Get specific help section, which explains the help tag conventions. It also helps to
                      Message 10 of 14 , Nov 2, 2008
                      • 0 Attachment
                        bill lam wrote:
                        > On Mon, 03 Nov 2008, Ben Schmidt wrote:
                        >> :set nomore
                        >>
                        >> :help 'more'
                        >
                        > Thank you Ben.
                        >
                        > I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                        > and :h 'cb' give different help pages, what is the rule for that
                        > quote?

                        The quote is for options. Just do

                        :help

                        all by itself and look at the 'Get specific help' section, which
                        explains the help tag conventions. It also helps to know that control
                        keys are shown like CTRL-A in help tags, and special keys like <BS>, and
                        tags involving such keys have underscores to separate the parts, e.g.
                        CTRL-W_<Up>.

                        :help <>

                        is also useful to look at.

                        Cheers,

                        Ben.




                        --~--~---------~--~----~------------~-------~--~----~
                        You received this message from the "vim_use" maillist.
                        For more information, visit http://www.vim.org/maillist.php
                        -~----------~----~----~----~------~----~------~--~---
                      • Ben Schmidt
                        ... And I should also mention ... If you scroll down a bit from there, you ll also find info on the useful ... Ben.
                        Message 11 of 14 , Nov 2, 2008
                        • 0 Attachment
                          Ben Schmidt wrote:
                          > bill lam wrote:
                          >> On Mon, 03 Nov 2008, Ben Schmidt wrote:
                          >>> :set nomore
                          >>>
                          >>> :help 'more'
                          >> Thank you Ben.
                          >>
                          >> I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                          >> and :h 'cb' give different help pages, what is the rule for that
                          >> quote?
                          >
                          > The quote is for options. Just do
                          >
                          > :help
                          >
                          > all by itself and look at the 'Get specific help' section, which
                          > explains the help tag conventions. It also helps to know that control
                          > keys are shown like CTRL-A in help tags, and special keys like <BS>, and
                          > tags involving such keys have underscores to separate the parts, e.g.
                          > CTRL-W_<Up>.
                          >
                          > :help <>
                          >
                          > is also useful to look at.

                          And I should also mention

                          :help :help

                          If you scroll down a bit from there, you'll also find info on the useful
                          :helpgrep and other such tips for using the help.

                          Ben.




                          --~--~---------~--~----~------------~-------~--~----~
                          You received this message from the "vim_use" maillist.
                          For more information, visit http://www.vim.org/maillist.php
                          -~----------~----~----~----~------~----~------~--~---
                        • bill lam
                          ... Thank you Tony and Ben. I once had read(browse) that :help help but can not notice the quotes are for an option. -- regards,
                          Message 12 of 14 , Nov 2, 2008
                          • 0 Attachment
                            On Mon, 03 Nov 2008, Tony Mechelynck wrote:

                            >
                            > On 03/11/08 07:39, bill lam wrote:
                            > > On Mon, 03 Nov 2008, Ben Schmidt wrote:
                            > >> :set nomore
                            > >>
                            > >> :help 'more'
                            > >
                            > > Thank you Ben.
                            > >
                            > > I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                            > > and :h 'cb' give different help pages, what is the rule for that
                            > > quote?
                            > >
                            >
                            > Quotes are for an option: ":help 'cb'" tells you what you can ":set cb" to.
                            >
                            > Nothing is for Normal-mode commands
                            >
                            > i_ for Insert, c_ for Command-line, v_ for Visual, : for Ex-commands, -
                            > for command-line switches, etc.
                            >
                            > See ":help help-context" on the very first page of help (where you land
                            > when you hit F1 in Vim).
                            >

                            Thank you Tony and Ben.

                            I once had read(browse) that :help help but can not notice the quotes
                            are for an option.

                            --
                            regards,
                            ====================================================
                            GPG key 1024D/4434BAB3 2008-08-24
                            gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

                            --~--~---------~--~----~------------~-------~--~----~
                            You received this message from the "vim_use" maillist.
                            For more information, visit http://www.vim.org/maillist.php
                            -~----------~----~----~----~------~----~------~--~---
                          • Tony Mechelynck
                            ... at the bottom of the list: Option :help textwidth Best regards, Tony. -- hundred-and-one symptoms of being an internet addict: 246. You use up
                            Message 13 of 14 , Nov 3, 2008
                            • 0 Attachment
                              On 03/11/08 08:41, bill lam wrote:
                              > On Mon, 03 Nov 2008, Tony Mechelynck wrote:
                              >
                              >> On 03/11/08 07:39, bill lam wrote:
                              >>> On Mon, 03 Nov 2008, Ben Schmidt wrote:
                              >>>> :set nomore
                              >>>>
                              >>>> :help 'more'
                              >>> Thank you Ben.
                              >>>
                              >>> I notice sometimes it need quote when :help, sometimes not? eg. :h cb
                              >>> and :h 'cb' give different help pages, what is the rule for that
                              >>> quote?
                              >>>
                              >> Quotes are for an option: ":help 'cb'" tells you what you can ":set cb" to.
                              >>
                              >> Nothing is for Normal-mode commands
                              >>
                              >> i_ for Insert, c_ for Command-line, v_ for Visual, : for Ex-commands, -
                              >> for command-line switches, etc.
                              >>
                              >> See ":help help-context" on the very first page of help (where you land
                              >> when you hit F1 in Vim).
                              >>
                              >
                              > Thank you Tony and Ben.
                              >
                              > I once had read(browse) that :help help but can not notice the quotes
                              > are for an option.
                              >

                              at the bottom of the list:

                              Option ' :help 'textwidth'


                              Best regards,
                              Tony.
                              --
                              hundred-and-one symptoms of being an internet addict:
                              246. You use up your free 100 hours in less than a week.

                              --~--~---------~--~----~------------~-------~--~----~
                              You received this message from the "vim_use" maillist.
                              For more information, visit http://www.vim.org/maillist.php
                              -~----------~----~----~----~------~----~------~--~---
                            Your message has been successfully submitted and would be delivered to recipients shortly.