Loading ...
Sorry, an error occurred while loading the content.
 

Re: Memory is used up when I replace "\t0\n" to "\n"

Expand Messages
  • Yakov Lerner
    ... 1. I don t understand why vim grows here (800MB and up)without giving memory overallocation error based on maxmem and maxmemtot . maxmem and
    Message 1 of 5 , Sep 15, 2009
      On Fri, Sep 11, 2009 at 06:57, Matt Wozniski <mjw@...> wrote:

      On Thu, Sep 10, 2009 at 4:13 AM, John Beckett wrote:
      >
      > Aleafs wrote:
      >> :1,$  s/\t0\n/\n/g

      Note that :% is a shortcut for :1,$

      > In a substitute, \n means two different things:
      > - In the pattern, it refers to a newline.
      > - In the replacement, it refers to a null byte (8 zero bits).
      >
      > You can see this at ':help :s' by following the first two links.
      >
      > In a replacement, '\r' inserts a newline. Yes, it's strange, and
      > no, I didn't try to work out why this issue caused the problem
      > you report.

      My guess: Vim is really, really bad at handling extremely long lines,
      and if the goal was to delete "the last column of a text file" (I'm
      assuming every line ended with "\t0") then this command told vim to
      turn the file into one 700kb long line.

      1. I don't understand why vim grows here (800MB and up)
      without giving memory overallocation error based on 'maxmem' and 'maxmemtot'. 
      'maxmem' and 'maxmemtot' have values of of 150524 (that's KB, or 150MB). 
      It's undo memory causes memory overuse here. 

      2. This problem of memory use here (and speed)is is with 'noswapfile' and 'undolevels', and it's solved by:

          :set noswapfile undolevel=-1,

      Then vim finishes the replacement (wait 1/2 minutes).
      And vim opens 700k-long file consisting of single line and many 0-bytes, in the first place, without problem.

      Testcase:

      1) 700k file consisting of single line and many 0-bytes
      perl -e 'printf "%8d %8d %8d %8d\n", $_,$_,$_,$_ for(1..20000)' >/tmp/1
      tr '\n' '\000' </tmp/1 >/tmp/2
      vim /tmp/2                 # no problem

      2) vim repalces newlines to 0-bytes without growing in memory, 700k file 20k lines:
      perl -e 'printf "%8d %8d %8d %8d\n", $_,$_,$_,$_ for(1..20000)' >/tmp/1
      vim /tmp/1
      :set noswapfile undolevel=-1
      :%s/\n/\n/               " finished in 30 sec, resul tis single 700k-long line

      3) without 'et noswapfile undolevel=-1', blows up vim to pieces:
      perl -e 'printf "%8d %8d %8d %8d\n", $_,$_,$_,$_ for(1..20000)' >/tmp/1
      vim /tmp/1
      :%s/\n/\n/ " grows beyond 1G in size and killed by OS, if you have patience
      Killed

      In scenario #3, I thought vim would report error based on 'maxmem' and 'maxmemtot', but
      it did not. My values of 'maxmem' and 'maxmemtot' are 150254 (=150MB), but why
      vim grew over 800MB in vm size without giving memory size error ?

      Yakov
       

      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_dev" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---

    Your message has been successfully submitted and would be delivered to recipients shortly.