Loading ...
Sorry, an error occurred while loading the content.

Vim memory use with long lines

Expand Messages
  • John Little
    Over on vim_use, in suggesting a solution to someone who wants to sort ... With this vim s heap grew to 512 MiB, and my 1 GiB system swapped somewhat for a
    Message 1 of 3 , Dec 3, 2010
    View Source
    • 0 Attachment
      Over on vim_use, in suggesting a solution to someone who wants to sort
      blocks of text by their titles, I ran the following in vim 7.3.55:

      :e $VIMRUNTIME/doc/eval.txt
      :save /tmp/eval.txt
      :v/^\d\+\. .*\*$/s/$/xyzzy
      :g/xyzzy/-j!

      With this vim's heap grew to 512 MiB, and my 1 GiB system swapped
      somewhat for a while, and it ran at 100% of cpu for a long time,
      having reported 8350 fewer lines. I closed the window and vim let me
      save the file.

      If I issue :syntax off before joining the lines, the command completes
      in a few seconds (if there's free memory) and does not run 100% cpu at
      all.

      So I see two problems:
      1) Profligate memory use. (I tried running with valgrind, and
      reported 536,211,001 bytes in use at exit, but the same, minor leaks
      as starting vim and immediately quitting.) Perhaps this is really a
      memory allocator deficiency, with vim repeatedly requesting slightly
      larger blocks to accommodate the growing line, and the allocator
      failing to consolidate the freed blocks.
      2) Syntax colouring for vim help doesn't cope with very long lines.
      Fair enough, one might say, but perhaps it could fail more gracefully.

      I don't really expect anything much to happen about these; I spent
      some time finding this stuff out so I thought I'd set it down, in case
      it added to the picture helpfully.

      (BTW, I followed with
      :sort /\d\+\. /
      :%s/xyzzy/\r/g
      :g/^\d\+\. .*\*$/-m .
      and it worked as intended.)

      Regards, John

      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • Bram Moolenaar
      ... Or it s text saved for undo. Try setting undolevels to -1. ... What is synmaxcol set to? ... -- For humans, honesty is a matter of degree. Engineers
      Message 2 of 3 , Dec 3, 2010
      View Source
      • 0 Attachment
        John Little wrote:

        > Over on vim_use, in suggesting a solution to someone who wants to sort
        > blocks of text by their titles, I ran the following in vim 7.3.55:
        >
        > :e $VIMRUNTIME/doc/eval.txt
        > :save /tmp/eval.txt
        > :v/^\d\+\. .*\*$/s/$/xyzzy
        > :g/xyzzy/-j!
        >
        > With this vim's heap grew to 512 MiB, and my 1 GiB system swapped
        > somewhat for a while, and it ran at 100% of cpu for a long time,
        > having reported 8350 fewer lines. I closed the window and vim let me
        > save the file.
        >
        > If I issue :syntax off before joining the lines, the command completes
        > in a few seconds (if there's free memory) and does not run 100% cpu at
        > all.
        >
        > So I see two problems:
        > 1) Profligate memory use. (I tried running with valgrind, and
        > reported 536,211,001 bytes in use at exit, but the same, minor leaks
        > as starting vim and immediately quitting.) Perhaps this is really a
        > memory allocator deficiency, with vim repeatedly requesting slightly
        > larger blocks to accommodate the growing line, and the allocator
        > failing to consolidate the freed blocks.

        Or it's text saved for undo. Try setting 'undolevels' to -1.

        > 2) Syntax colouring for vim help doesn't cope with very long lines.
        > Fair enough, one might say, but perhaps it could fail more gracefully.

        What is 'synmaxcol' set to?

        > I don't really expect anything much to happen about these; I spent
        > some time finding this stuff out so I thought I'd set it down, in case
        > it added to the picture helpfully.
        >
        > (BTW, I followed with
        > :sort /\d\+\. /
        > :%s/xyzzy/\r/g
        > :g/^\d\+\. .*\*$/-m .
        > and it worked as intended.)
        >
        > Regards, John

        --
        For humans, honesty is a matter of degree. Engineers are always honest in
        matters of technology and human relationships. That's why it's a good idea
        to keep engineers away from customers, romantic interests, and other people
        who can't handle the truth.
        (Scott Adams - The Dilbert principle)

        /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
        /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
        \\\ an exciting new programming language -- http://www.Zimbu.org ///
        \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • John Little
        ... I set undolevels=-1 and the heap remained at about 7 MiB. ... 3000. Trying various settings for synmaxcol, :syntax on takes: 100 10 s 200 24 s 500
        Message 3 of 3 , Dec 4, 2010
        View Source
        • 0 Attachment
          > John Little wrote:
          > > Over on vim_use, in suggesting a solution...
          > > With this vim's heap grew to 512 MiB...
          >
          > > So I see two problems:
          > > 1) Profligate memory use...

          Bram responded:
          > Or it's text saved for undo.  Try setting 'undolevels' to -1.

          I set undolevels=-1 and the heap remained at about 7 MiB.

          > > 2) Syntax colouring for vim help doesn't cope with very long lines.

          Bram responded:
          > What is 'synmaxcol' set to?

          3000. Trying various settings for synmaxcol, :syntax on takes:
          100 10 s
          200 24 s
          500 56 s
          1000 114 s

          (this is on a 2.2 GHz dual core Athlon 64, Kubuntu).

          There are syntax files with very long lines, f.ex. php.vim, that have
          no trouble, so reducing synmaxcol for help files generally doesn't
          seem warranted.

          Regards, John

          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        Your message has been successfully submitted and would be delivered to recipients shortly.