Loading ...
Sorry, an error occurred while loading the content.

Re: Vim slow after big count insert

Expand Messages
  • LCD 47
    ... [...] ... [...] Right, what you re seeing here is Vim backtracking to find a blank to split that N * Hello line. Since there is no such blank, you get
    Message 1 of 17 , Jul 20, 2013
    • 0 Attachment
      On 19 July 2013, Mike Williams <mike.williams@...> wrote:
      > On 19/07/2013 15:52, Mike Williams wrote:
      > >On 19/07/2013 12:18, Dimitar DIMITROV wrote:
      > >>Hi,
      > >>
      > >>Did a search on the vim_dev archives but couldn't find anything
      > >>related to this. Sorry if this is redundant. Basically vim is
      > >>exponentially slow after 1000000iHello <esc> as mentionned in this
      > >>link:
      > >>http://www.galexander.org/vim_sucks.html
      > >
      > >A quick run over with a profiler seems to show most of the time is
      > >being spent in vim_strchr() called from has_format_option(), called
      > >from internal_format(). If I first do :set paste then it is a lot
      > >quicker - for 4000iHello<ESC> it goes from 16s down to 1s.
      [...]
      > Textwidth. Darn sight quicker with it set to 0, wish I had
      > remembered that. Another factor of ~2 quicker for my final times,
      > 16000iHello<ESC> now takes ~4s.
      [...]

      Right, what you're seeing here is Vim backtracking to find a blank
      to split that N * "Hello" line. Since there is no such blank, you get
      len("Hello") * N * (N + 1) / 2 failed comparisons, which is exactly the
      O(N^2) behaviour claimed in the article mentioned above.

      If you replace your test by, say, "16000iHello <ESC>" (with a space
      after "Hello"), you get linear behaviour, namely len("Hello") * N failed
      comparisons.

      I believe this can be fixed with a counter that means something
      along the lines of: "this line is longer than &tw, and it has no
      breaking point for the first X characters". Then X would be updated
      every time more text is appended to that line.

      On 19 July 2013, Marc Weber <marco-oweber@...> wrote:
      > Let's discuss the use case, first. Is there one ?

      Yes, there is: you run through that piece of code every time you
      press "p" to paste text. It hits you particularly hard when you edit
      files with very long lines.

      > If not let's forgett about it ..
      [...]

      Perhaps we should try to understand the problem before dismissing it
      as harmless?

      /lcd

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • Mike Williams
      ... Yup to all that. And there is another O(n^2) operation is getvcol() rescanning the line after each insert. ... Indeed. Part of the problem is that most
      Message 2 of 17 , Jul 21, 2013
      • 0 Attachment
        On 20/07/2013 10:13, LCD 47 wrote:
        > On 19 July 2013, Mike Williams <mike.williams@...> wrote:
        >> On 19/07/2013 15:52, Mike Williams wrote:
        >>> On 19/07/2013 12:18, Dimitar DIMITROV wrote:
        >>>> Hi,
        >>>>
        >>>> Did a search on the vim_dev archives but couldn't find anything
        >>>> related to this. Sorry if this is redundant. Basically vim is
        >>>> exponentially slow after 1000000iHello <esc> as mentionned in this
        >>>> link:
        >>>> http://www.galexander.org/vim_sucks.html
        >>>
        >>> A quick run over with a profiler seems to show most of the time is
        >>> being spent in vim_strchr() called from has_format_option(), called
        >> >from internal_format(). If I first do :set paste then it is a lot
        >>> quicker - for 4000iHello<ESC> it goes from 16s down to 1s.
        > [...]
        >> Textwidth. Darn sight quicker with it set to 0, wish I had
        >> remembered that. Another factor of ~2 quicker for my final times,
        >> 16000iHello<ESC> now takes ~4s.
        > [...]
        >
        > Right, what you're seeing here is Vim backtracking to find a blank
        > to split that N * "Hello" line. Since there is no such blank, you get
        > len("Hello") * N * (N + 1) / 2 failed comparisons, which is exactly the
        > O(N^2) behaviour claimed in the article mentioned above.
        >
        > If you replace your test by, say, "16000iHello <ESC>" (with a space
        > after "Hello"), you get linear behaviour, namely len("Hello") * N failed
        > comparisons.
        >
        > I believe this can be fixed with a counter that means something
        > along the lines of: "this line is longer than &tw, and it has no
        > breaking point for the first X characters". Then X would be updated
        > every time more text is appended to that line.

        Yup to all that. And there is another O(n^2) operation is getvcol()
        rescanning the line after each insert.

        > On 19 July 2013, Marc Weber <marco-oweber@...> wrote:
        >> Let's discuss the use case, first. Is there one ?
        >
        > Yes, there is: you run through that piece of code every time you
        > press "p" to paste text. It hits you particularly hard when you edit
        > files with very long lines.
        >
        >> If not let's forgett about it ..
        > [...]
        >
        > Perhaps we should try to understand the problem before dismissing it
        > as harmless?

        Indeed. Part of the problem is that most computers these days have
        ridiculous amounts of processing power and storage that slow algorithms
        for small to medium amounts of data appear near instantaneous. On say
        an Atom or Arm based netbook then the problem becomes more apparent.
        Getting a faster machine is not always the solution.

        Mike
        --
        Born naked, wet and hungry... and it gets worse?

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
      • Charles Campbell
        ... I tried this problem with both 1000000iHello 1000000iHello In both cases ctrl-c worked just fine to break the operation. I used vim 7.4a.35 on a
        Message 3 of 17 , Jul 22, 2013
        • 0 Attachment
          Dimitar DIMITROV wrote:
          > > > There is no use case
          > > If you do something stupid by accident most vim operations can be
          > > aborted by ctrl-c (exception: python, rbuy, .. scripts)
          >
          > Try to abort it you will see the success you have.
          >
          > > So there is still nothing to fix or talk about unless there is a use
          > > case.
          > >
          > > Marc Weber
          I tried this problem with both

          1000000iHello<esc>
          1000000iHello <esc>

          In both cases ctrl-c worked just fine to break the operation.

          I used vim 7.4a.35 on a linux system when trying this.

          Regards,
          C Campbell

          --
          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_dev" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • Nikolay Pavlov
          ... This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and
          Message 4 of 17 , Jul 22, 2013
          • 0 Attachment


            On Jul 20, 2013 2:28 AM, "Gary Johnson" <garyjohn@...> wrote:
            >
            > On 2013-07-19, Dimitar DIMITROV wrote:
            >
            > Hi Dimitar,
            >
            > It would be a big help to those of us with threading mail readers if
            > you would be sure that your replies include "Re: " at the start of
            > the Subject.

            This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and *this* field makes mail readers able to arrange messages correctly.

            > Thanks,
            > Gary
            >
            > --
            > --
            > You received this message from the "vim_dev" maillist.
            > Do not top-post! Type your reply below the text you are replying to.
            > For more information, visit http://www.vim.org/maillist.php
            >
            > ---
            > You received this message because you are subscribed to the Google Groups "vim_dev" group.
            > To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            > For more options, visit https://groups.google.com/groups/opt_out.
            >
            >

            --
            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
             
            ---
            You received this message because you are subscribed to the Google Groups "vim_dev" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
             
             
          • James McCoy
            ... That depends on the mail reader. Even with MUAs that follow the Message-ID/In-Reply-To chains, it s not uncommon to treat a topic change as a new thread.
            Message 5 of 17 , Jul 22, 2013
            • 0 Attachment
              On Tue, Jul 23, 2013 at 07:39:12AM +0400, Nikolay Pavlov wrote:
              > On Jul 20, 2013 2:28 AM, "Gary Johnson" <garyjohn@...> wrote:
              > >
              > > On 2013-07-19, Dimitar DIMITROV wrote:
              > >
              > > Hi Dimitar,
              > >
              > > It would be a big help to those of us with threading mail readers if
              > > you would be sure that your replies include "Re: " at the start of
              > > the Subject.
              >
              > This has nothing to do with threads. There is a special field added to
              > messages when you reply that contains unique ID of the message being
              > replied to and *this* field makes mail readers able to arrange messages
              > correctly.

              That depends on the mail reader. Even with MUAs that follow the
              Message-ID/In-Reply-To chains, it's not uncommon to treat a topic change
              as a new thread. For ones that don't honor the IDs, the subject is
              usually used instead.

              Regardless, Dimitar's emails don't even have the In-Reply-To header set.
              The X-Mailer header claims to be YahooMailWebService, so I doubt there's
              much that he can do to fix it short of deciding to use a more standard,
              standalone MUA instead of (I assume) their web interface.

              Cheers,
              --
              James
              GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy <jamessan@...>

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
            • Gary Johnson
              ... I m well aware of those fields, and my mail reader (mutt) handles them quite well. It is also capable of threading messages that lack those fields and
              Message 6 of 17 , Jul 22, 2013
              • 0 Attachment
                On 2013-07-23, Nikolay Pavlov wrote:
                >
                > On Jul 20, 2013 2:28 AM, "Gary Johnson" wrote:
                > >
                > > On 2013-07-19, Dimitar DIMITROV wrote:
                > >
                > > Hi Dimitar,
                > >
                > > It would be a big help to those of us with threading mail readers if
                > > you would be sure that your replies include "Re: " at the start of
                > > the Subject.
                >
                > This has nothing to do with threads. There is a special field added to messages
                > when you reply that contains unique ID of the message being replied to and
                > *this* field makes mail readers able to arrange messages correctly.

                I'm well aware of those fields, and my mail reader (mutt) handles
                them quite well. It is also capable of threading messages that lack
                those fields and have only "Re: " in front of the original subject
                to go by. Dimitar's messages did not contain those headers, though,
                and since his Subject lines didn't even include the "Re: ", I
                assumed it would be a stretch for his mail client to do any more
                than allow him to add the "Re: ".

                Regards,
                Gary

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              • Nikolay Pavlov
                ... messages ... and ... Good to know something new. One my mailer ignores everything but special header, another splits threads on theme change (I did not
                Message 7 of 17 , Jul 23, 2013
                • 0 Attachment


                  On Jul 23, 2013 9:24 AM, "Gary Johnson" <garyjohn@...> wrote:
                  >
                  > On 2013-07-23, Nikolay Pavlov wrote:
                  > >
                  > > On Jul 20, 2013 2:28 AM, "Gary Johnson" wrote:
                  > > >
                  > > > On 2013-07-19, Dimitar DIMITROV wrote:
                  > > >
                  > > > Hi Dimitar,
                  > > >
                  > > > It would be a big help to those of us with threading mail readers if
                  > > > you would be sure that your replies include "Re: " at the start of
                  > > > the Subject.
                  > >
                  > > This has nothing to do with threads. There is a special field added to messages
                  > > when you reply that contains unique ID of the message being replied to and
                  > > *this* field makes mail readers able to arrange messages correctly.
                  >
                  > I'm well aware of those fields, and my mail reader (mutt) handles
                  > them quite well.  It is also capable of threading messages that lack
                  > those fields and have only "Re: " in front of the original subject
                  > to go by.  Dimitar's messages did not contain those headers, though,
                  > and since his Subject lines didn't even include the "Re: ", I
                  > assumed it would be a stretch for his mail client to do any more
                  > than allow him to add the "Re: ".

                  Good to know something new. One my mailer ignores everything but special header, another splits threads on theme change (I did not realize I was getting separate threads for this very reason though), but does not join messages in threads if they lack field in question. So I assumed this field is required for all mailers without performing more research.

                  > Regards,
                  > Gary
                  >
                  > --
                  > --
                  > You received this message from the "vim_dev" maillist.
                  > Do not top-post! Type your reply below the text you are replying to.
                  > For more information, visit http://www.vim.org/maillist.php
                  >
                  > ---
                  > You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  > To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  > For more options, visit https://groups.google.com/groups/opt_out.
                  >
                  >

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php
                   
                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                   
                   
                • Christian Brabandt
                  Hi Mike! ... Are you going to create a patch for that? ... Not only that, but getvcol() is called twice from the edit() function. Once from
                  Message 8 of 17 , Jul 30, 2013
                  • 0 Attachment
                    Hi Mike!

                    On So, 21 Jul 2013, Mike Williams wrote:

                    > On 20/07/2013 10:13, LCD 47 wrote:
                    > > I believe this can be fixed with a counter that means something
                    > >along the lines of: "this line is longer than &tw, and it has no
                    > >breaking point for the first X characters". Then X would be updated
                    > >every time more text is appended to that line.

                    Are you going to create a patch for that?

                    > Yup to all that. And there is another O(n^2) operation is getvcol()
                    > rescanning the line after each insert.

                    Not only that, but getvcol() is called twice from the edit() function.
                    Once from validate_cursor_col() at line 695 and once from
                    validate_cursor() at line 725. It seems like that either one of those
                    calls could be saved.

                    regards,
                    Christian
                    --
                    Wie man sein Kind nicht nennen sollte:
                    Hans Dampf

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/groups/opt_out.
                  Your message has been successfully submitted and would be delivered to recipients shortly.