Loading ...
Sorry, an error occurred while loading the content.

Why is Vimscript/VimL much slower than Python when traversing long lists?

Expand Messages
  • Vlad Irnov
    The following is a simple test that compares the times it takes to traverse a long list with FOR loop in Vimscript and in Python 2.x:
    Message 1 of 8 , Mar 15, 2013
    • 0 Attachment
      The following is a simple test that compares the times it takes to
      traverse a long list with FOR loop in Vimscript and in Python 2.x:

      "-----BEGIN------------------------------
      let start = reltime()
      for i in range(1000000)
      endfor
      let time_vim = reltimestr(reltime(start))

      let start = reltime()
      python << EOF
      for i in range(1000000): pass
      EOF
      let time_py = reltimestr(reltime(start))

      echo 'Vim: ' time_vim
      echo 'Python:' time_py
      "-----END--------------------------------

      The times I get (on an ancient computer):
      Vim: 9.2 +- 0.2
      Python: 0.26 +- 0.01

      The exact times vary depending on how the test is done, but Vimscript
      is invariably 30-45 times slower than Python. Why is there such a big
      difference in performance?

      --
      --
      You received this message from the "vim_use" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_use" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • Marc Weber
      VimL is known to be slow. If you know that you need speed use python or external tools. Why is it the way it is? I don t know. Eventually wait for better
      Message 2 of 8 , Mar 15, 2013
      • 0 Attachment
        VimL is known to be slow.

        If you know that you need speed use python or external tools.

        Why is it the way it is? I don't know.

        Eventually wait for better replies.

        I can recommend using a disnict .py file for writing python because then
        you get all python support (syntax, completion, etc).

        If you want to learn how a .py file can be found by path have a look at
        github.com/MarcWeber/scion-backend-vim

        let s:self=expand('<sfile>:h')

        You may also want to use a str quoting function like this:

        def vimQuote(s):
        return '"%s"' % s.replace("\\","\\\\").replace('"', '\\"').replace("\n", "\\n")

        because python's to str comes only close (maybe I should retry the to
        json implementation and find out when whether it fails)

        Marc Weber

        --
        --
        You received this message from the "vim_use" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_use" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
      • John Little
        Well, I know squat about Python but I presume it has a compilation phase like Perl, which compiles to op codes which are executed without, f. ex., variable
        Message 3 of 8 , Mar 15, 2013
        • 0 Attachment
          Well, I know squat about Python but I presume it has a compilation phase like Perl, which compiles to "op codes" which are executed without, f. ex., variable look ups. Vim uses a pure interpreter.

          Vim script is "fast enough", and has the virtue of running anywhere vim does (well, not quite; anywhere a "normal" non-gui build does) with few dependencies, IIUC a C compiler and the standard library.

          Regards, John Little

          --
          --
          You received this message from the "vim_use" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_use" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • Vlad Irnov
          ... You are probably right. Python does compile the source code into bytecode. It s interesting that such basic operation as for loop gets 30-fold
          Message 4 of 8 , Mar 15, 2013
          • 0 Attachment
            On 3/15/13, John Little <John.B.Little@...> wrote:
            > Well, I know squat about Python but I presume it has a compilation phase
            > like Perl, which compiles to "op codes" which are executed without, f. ex.,
            > variable look ups. Vim uses a pure interpreter.
            >
            > Vim script is "fast enough", and has the virtue of running anywhere vim does
            > (well, not quite; anywhere a "normal" non-gui build does) with few
            > dependencies, IIUC a C compiler and the standard library.
            >
            > Regards, John Little
            >

            You are probably right. Python does compile the source code into
            bytecode. It's interesting that such basic operation as "for" loop
            gets >30-fold performance boost.

            Regards,
            Vlad

            --
            --
            You received this message from the "vim_use" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php

            ---
            You received this message because you are subscribed to the Google Groups "vim_use" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
          • Salman Halim
            ... It s entirely possible that the loop in question was simply left out as a compiler optimization. Modern compilers can detect no-op loops and unchanging
            Message 5 of 8 , Mar 15, 2013
            • 0 Attachment
              On Friday, March 15, 2013, Vlad Irnov wrote:
              On 3/15/13, John Little <John.B.Little@...> wrote:
              > Well, I know squat about Python but I presume it has a compilation phase
              > like Perl, which compiles to "op codes" which are executed without, f. ex.,
              > variable look ups. Vim uses a pure interpreter.
              >
              > Vim script is "fast enough", and has the virtue of running anywhere vim does
              > (well, not quite; anywhere a "normal" non-gui build does) with few
              > dependencies, IIUC a C compiler and the standard library.
              >
              > Regards, John Little
              >

              You are probably right. Python does compile the source code into
              bytecode. It's interesting that such basic operation as "for" loop
              gets >30-fold performance boost.

              Regards,
              Vlad

              --
              --
              You received this message from the "vim_use" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_use" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.


              It's entirely possible that the loop in question was simply left out as a compiler optimization. Modern compilers can detect no-op loops and unchanging assignments and take these things out of the compiled code. 

              Salman 


              --
              سلمان حلیم

              --
              --
              You received this message from the "vim_use" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php
               
              ---
              You received this message because you are subscribed to the Google Groups "vim_use" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
               
               
            • Vlad Irnov
              On 3/15/13, Salman Halim wrote: ... This is not the case here. It s easy to check that the time it takes to run the no-op Python loop
              Message 6 of 8 , Mar 18, 2013
              • 0 Attachment
                On 3/15/13, Salman Halim <salmanhalim@...> wrote:
                ...
                > It's entirely possible that the loop in question was simply left out as a
                > compiler optimization. Modern compilers can detect no-op loops and
                > unchanging assignments and take these things out of the compiled code.
                >
                > Salman
                >
                >
                > --
                > سلمان حلیم

                This is not the case here. It's easy to check that the time it takes to
                run the no-op Python loop is proportional to the number of steps, which
                means it is not skipped:

                let start = reltime()
                python for i in range(10000000): pass
                echo reltimestr(reltime(start))
                let start = reltime()
                python for i in range(100000): pass
                echo reltimestr(reltime(start))

                I also tried adding some code to the body of each loop in my first
                example and still got about 30-fold difference.

                Regards,
                Vlad

                --
                --
                You received this message from the "vim_use" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_use" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              • Vlad Irnov
                ... Saying that VimL is slow is pointless without explaining exactly what is slow(er) and by how much. I am not aware of any actual comparisons of VimL
                Message 7 of 8 , Mar 18, 2013
                • 0 Attachment
                  On 3/15/13, Marc Weber <marco-oweber@...> wrote:
                  > VimL is known to be slow.

                  Saying that VimL is slow is pointless without explaining exactly what is
                  slow(er) and by how much. I am not aware of any actual comparisons of
                  VimL performance vs other scripting languages. My own conclusion from
                  very few crude tests is that VimL can be much slower than Python when
                  dealing with long lists and strings, 20 to 100 times slower. In terms of
                  actual execution times this only begins to matter with sizes >100000
                  items. Using map(), filter(), etc. instead of "for" loop help somewhat.
                  Not everything is slow, :g/pattern/... is fast.


                  > If you know that you need speed use python or external tools.
                  >
                  > Why is it the way it is? I don't know.
                  >
                  > Eventually wait for better replies.
                  >
                  > I can recommend using a disnict .py file for writing python because then
                  > you get all python support (syntax, completion, etc).
                  >
                  > If you want to learn how a .py file can be found by path have a look at
                  > github.com/MarcWeber/scion-backend-vim
                  >
                  > let s:self=expand('<sfile>:h')

                  I use Python a lot in my VOoM plugin. You are absolutely right that any
                  substantial Python code should be in distinct .py files instead of
                  inside .vim files.


                  > You may also want to use a str quoting function like this:
                  >
                  > def vimQuote(s):
                  > return '"%s"' % s.replace("\\","\\\\").replace('"', '\\"').replace("\n",
                  > "\\n")
                  >
                  > because python's to str comes only close (maybe I should retry the to
                  > json implementation and find out when whether it fails)
                  >
                  > Marc Weber

                  I usually do
                  "'%s'" % s.replace("'","''")

                  Regards,
                  Vlad

                  --
                  --
                  You received this message from the "vim_use" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php

                  ---
                  You received this message because you are subscribed to the Google Groups "vim_use" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                • Marc Weber
                  ... I ve tried hacking a delphi completion once. Even though using aggressive caching in VimL completion was slower than 1sec. The other thing I tried is
                  Message 8 of 8 , Mar 18, 2013
                  • 0 Attachment
                    Excerpts from Vlad Irnov's message of Mon Mar 18 11:16:51 +0100 2013:
                    > On 3/15/13, Marc Weber <marco-oweber@...> wrote:
                    > > VimL is known to be slow.
                    > Saying that VimL is slow is pointless without explaining exactly what is
                    > slow(er) and by how much.

                    I've tried hacking a delphi completion once. Even though using
                    aggressive caching in VimL completion was slower than 1sec.

                    The other thing I tried is parsing viml files, to implement a tag like
                    goto function feature and more - it also takes more than 15 sec to parse
                    and cache all .vim files the first time .

                    Such implementations horribly fail on large source bases such as flex
                    sdk (actionscript).

                    Sorry - I don't know what exactly is slow - and I don't care.

                    My impression of UltiSnips vs snipmate (viml) is the same: that the
                    python implementation is faster.

                    No, its not an accurate benchmark. And to be honest I don't care.
                    Same about YouCompleteMe vs NeoComplCache (YouCompleteMe is written in C
                    for a strong reason: speed).

                    Applying regex to many lines is likely to be one point - but there might
                    be more.

                    Whenever you have huge amounts of data don't use viml unless you can use
                    viml builtins (this also applies to :g :v vs :!%grep etc)

                    You're right that it could be possible to optimize VimL and identify the
                    problems (split or join was one which got fixed). But I don't have time
                    to do so - if other solutions already work.

                    Marc Weber

                    --
                    --
                    You received this message from the "vim_use" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_use" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@....
                    For more options, visit https://groups.google.com/groups/opt_out.
                  Your message has been successfully submitted and would be delivered to recipients shortly.