Loading ...
Sorry, an error occurred while loading the content.

Possible problem with sudden shortening of very large files by an external process

Expand Messages
  • Gary Bickford
    I often misuse vim to view log files while debugging batch jobs. It is very handy for moving around in the file, viewing the colorized content and searching
    Message 1 of 4 , Feb 26, 2010
    • 0 Attachment
      I often misuse vim to view log files while debugging batch jobs. It is
      very handy for moving around in the file, viewing the colorized content
      and searching for relevant text. These log files can be from several
      thousand to millions of lines. Vim does a remarkably good job of
      handling these files. However, sometimes it has a problem when the
      batch job is re-run while I am viewing the file in vim, and the log file
      is replaced by a much shorter one.

      When vim tries to reload the file, sometimes I get a very long series of
      red warning messages - I can't repeat the event just now, so can't
      provide the content of the warning, but I think it has to do with
      missing lines.

      This succession of warnings can be inconvenient when there are several
      million to report, or even a few thousand! And very occasionally vim
      hangs (apparently), or just gives up the ghost and dies. I think
      problems are more likely when the 'present location' is far down the
      file.

      Considering the abuse I am giving vim this is within the range of
      acceptable behavior, but I thought it was worth mentioning as an issue
      anyway. From the behavior, I speculate that vim has found the present
      line to be after the end of file, is trying to move the line pointer
      upwards one line at a time, and is finding no data there each time.
      --
      "He that breaks a thing to find out what it is has left the path of
      wisdom."-- J.R.R. Tolkien
      - But where does that leave particle physics?

      --
      You received this message from the "vim_use" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • Bram Moolenaar
      ... It would help a lot to know what the text of that error message is. If it is ml_get: cannot find line {N} then the cause might be hard to find. A stack
      Message 2 of 4 , Feb 26, 2010
      • 0 Attachment
        Gary Bickford wrote:

        > I often misuse vim to view log files while debugging batch jobs. It is
        > very handy for moving around in the file, viewing the colorized content
        > and searching for relevant text. These log files can be from several
        > thousand to millions of lines. Vim does a remarkably good job of
        > handling these files. However, sometimes it has a problem when the
        > batch job is re-run while I am viewing the file in vim, and the log file
        > is replaced by a much shorter one.
        >
        > When vim tries to reload the file, sometimes I get a very long series of
        > red warning messages - I can't repeat the event just now, so can't
        > provide the content of the warning, but I think it has to do with
        > missing lines.
        >
        > This succession of warnings can be inconvenient when there are several
        > million to report, or even a few thousand! And very occasionally vim
        > hangs (apparently), or just gives up the ghost and dies. I think
        > problems are more likely when the 'present location' is far down the
        > file.
        >
        > Considering the abuse I am giving vim this is within the range of
        > acceptable behavior, but I thought it was worth mentioning as an issue
        > anyway. From the behavior, I speculate that vim has found the present
        > line to be after the end of file, is trying to move the line pointer
        > upwards one line at a time, and is finding no data there each time.

        It would help a lot to know what the text of that error message is.

        If it is " ml_get: cannot find line {N}" then the cause might be hard to
        find. A stack trace would be helpful then.

        --
        ZOOT: I'm afraid our life must seem very dull and quiet compared to yours.
        We are but eightscore young blondes, all between sixteen and
        nineteen-and-a-half, cut off in this castle, with no one to protect us.
        Oooh. It is a lonely life ... bathing ... dressing ... undressing ...
        making exciting underwear....
        "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

        /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
        /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
        \\\ download, build and distribute -- http://www.A-A-P.org ///
        \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

        --
        You received this message from the "vim_use" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • Gary Bickford
        ... That is the error message - E315 ml_get: cannot find line (N). I note that if I type q , when it waits for me to hit enter, I don t have to hit Enter
        Message 3 of 4 , Mar 1, 2010
        • 0 Attachment
          Bram Moolenaar wrote:
          > Gary Bickford wrote:
          >
          >
          >> I often misuse vim to view log files while debugging batch jobs. It is
          >> very handy for moving around in the file, viewing the colorized content
          >> and searching for relevant text. These log files can be from several
          >> thousand to millions of lines. Vim does a remarkably good job of
          >> handling these files. However, sometimes it has a problem when the
          >> batch job is re-run while I am viewing the file in vim, and the log file
          >> is replaced by a much shorter one.
          >>
          >> When vim tries to reload the file, sometimes I get a very long series of
          >> red warning messages - I can't repeat the event just now, so can't
          >> provide the content of the warning, but I think it has to do with
          >> missing lines.
          >>
          >> This succession of warnings can be inconvenient when there are several
          >> million to report, or even a few thousand! And very occasionally vim
          >> hangs (apparently), or just gives up the ghost and dies. I think
          >> problems are more likely when the 'present location' is far down the
          >> file.
          >>
          >> Considering the abuse I am giving vim this is within the range of
          >> acceptable behavior, but I thought it was worth mentioning as an issue
          >> anyway. From the behavior, I speculate that vim has found the present
          >> line to be after the end of file, is trying to move the line pointer
          >> upwards one line at a time, and is finding no data there each time.
          >>
          >
          > It would help a lot to know what the text of that error message is.
          >
          > If it is " ml_get: cannot find line {N}" then the cause might be hard to
          > find. A stack trace would be helpful then.
          >
          >
          That is the error message - E315 ml_get: cannot find line (N). I note
          that if I type 'q', when it waits for me to hit enter, I don't have to
          hit Enter after each line, it just spews this error message with
          increasing line numbers until it gets to the end or dies with a segfault.

          I am not at work so I can't run any tests today - sorry for the slow
          response cycle. Since I don't write C, I haven't had to deal with
          segfaults for a long time. To get a stack trace, do I need to run vim
          under gdb?

          --
          You received this message from the "vim_use" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        • Dominique Pellé
          ... Hi E315 error indicates an internal error according to :help 315 . It s definitely worth reporting. Which version of Vim are you using? Latest version is
          Message 4 of 4 , Mar 1, 2010
          • 0 Attachment
            Gary Bickford <gar37bic@...> wrote:

            > Bram Moolenaar wrote:
            >>
            >> Gary Bickford wrote:
            >>
            >>
            >>>
            >>> I often misuse vim to view log files while debugging batch jobs.  It is
            >>> very handy for moving around in the file, viewing the colorized content
            >>> and searching for relevant text.  These log files can be from several
            >>> thousand to millions of lines.  Vim does a remarkably good job of
            >>> handling these files.  However, sometimes it has a problem when the
            >>> batch job is re-run while I am viewing the file in vim, and the log file
            >>> is replaced by a much shorter one.
            >>>
            >>> When vim tries to reload the file, sometimes I get a very long series of
            >>> red warning messages - I can't repeat the event just now, so can't
            >>> provide the content of the warning, but I think it has to do with
            >>> missing lines.
            >>>
            >>> This succession of warnings can be inconvenient when there are several
            >>> million to report, or even a few thousand!  And very occasionally vim
            >>> hangs (apparently), or just gives up the ghost and dies.  I think
            >>> problems are more likely when the 'present location' is far down the
            >>> file.
            >>>
            >>> Considering the abuse I am giving vim this is within the range of
            >>> acceptable behavior, but I thought it was worth mentioning as an issue
            >>> anyway.  From the behavior, I speculate that vim has found the present
            >>> line to be after the end of file, is trying to move the line pointer
            >>> upwards one line at a time, and is finding no data there each time.
            >>>
            >>
            >> It would help a lot to know what the text of that error message is.
            >>
            >> If it is " ml_get: cannot find line {N}" then the cause might be hard to
            >> find.  A stack trace would be helpful then.
            >>
            >>
            >
            > That is the error message - E315 ml_get: cannot find line (N).  I note that
            > if I type 'q', when it waits for me to hit enter, I don't have to hit Enter
            > after each line, it just spews this error message with increasing line
            > numbers until it gets to the end or dies with a segfault.
            >
            > I am not at work so I can't run any tests today - sorry for the slow
            > response cycle.  Since I don't write C, I haven't had to deal with segfaults
            > for a long time.  To get a stack trace, do I need to run vim under gdb?

            Hi

            E315 error indicates an internal error according to ":help 315".
            It's definitely worth reporting.

            Which version of Vim are you using? Latest version is 7.2.377.
            You can see the version with ":version".

            Vim only loads the file at startup so even if the file is being changed
            be another process while being edited in Vim, it should not matter I
            would think. Do you remember what you were doing in Vim which
            could trigger this bug? What plugins did you install?

            Looking at ftp://ftp.vim.org/pub/vim/patches/7.2/README, I see
            several recent patches which fixed E315 errors:

            7.2.215 ml_get error when using ":vimgrep"
            7.2.226 ml_get error after deleting the last line
            7.2.375 ml_get errors when using ":bprevious" in a BufEnter autocmd
            7.2.376 ml_get error when using SiSU syntax

            So bug may have already been fixed. I would suggest to
            download & compile the latest version of Vim if you can.

            -- Dominique

            --
            You received this message from the "vim_use" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
          Your message has been successfully submitted and would be delivered to recipients shortly.