Loading ...
Sorry, an error occurred while loading the content.

Re: Vim slow after big count insert

Expand Messages
  • Gary Johnson
    On 2013-07-19, Dimitar DIMITROV wrote: Hi Dimitar, It would be a big help to those of us with threading mail readers if you would be sure that your replies
    Message 1 of 17 , Jul 19, 2013
    • 0 Attachment
      On 2013-07-19, Dimitar DIMITROV wrote:

      Hi Dimitar,

      It would be a big help to those of us with threading mail readers if
      you would be sure that your replies include "Re: " at the start of
      the Subject.

      Thanks,
      Gary

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • LCD 47
      ... [...] ... [...] Right, what you re seeing here is Vim backtracking to find a blank to split that N * Hello line. Since there is no such blank, you get
      Message 2 of 17 , Jul 20, 2013
      • 0 Attachment
        On 19 July 2013, Mike Williams <mike.williams@...> wrote:
        > On 19/07/2013 15:52, Mike Williams wrote:
        > >On 19/07/2013 12:18, Dimitar DIMITROV wrote:
        > >>Hi,
        > >>
        > >>Did a search on the vim_dev archives but couldn't find anything
        > >>related to this. Sorry if this is redundant. Basically vim is
        > >>exponentially slow after 1000000iHello <esc> as mentionned in this
        > >>link:
        > >>http://www.galexander.org/vim_sucks.html
        > >
        > >A quick run over with a profiler seems to show most of the time is
        > >being spent in vim_strchr() called from has_format_option(), called
        > >from internal_format(). If I first do :set paste then it is a lot
        > >quicker - for 4000iHello<ESC> it goes from 16s down to 1s.
        [...]
        > Textwidth. Darn sight quicker with it set to 0, wish I had
        > remembered that. Another factor of ~2 quicker for my final times,
        > 16000iHello<ESC> now takes ~4s.
        [...]

        Right, what you're seeing here is Vim backtracking to find a blank
        to split that N * "Hello" line. Since there is no such blank, you get
        len("Hello") * N * (N + 1) / 2 failed comparisons, which is exactly the
        O(N^2) behaviour claimed in the article mentioned above.

        If you replace your test by, say, "16000iHello <ESC>" (with a space
        after "Hello"), you get linear behaviour, namely len("Hello") * N failed
        comparisons.

        I believe this can be fixed with a counter that means something
        along the lines of: "this line is longer than &tw, and it has no
        breaking point for the first X characters". Then X would be updated
        every time more text is appended to that line.

        On 19 July 2013, Marc Weber <marco-oweber@...> wrote:
        > Let's discuss the use case, first. Is there one ?

        Yes, there is: you run through that piece of code every time you
        press "p" to paste text. It hits you particularly hard when you edit
        files with very long lines.

        > If not let's forgett about it ..
        [...]

        Perhaps we should try to understand the problem before dismissing it
        as harmless?

        /lcd

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
      • Mike Williams
        ... Yup to all that. And there is another O(n^2) operation is getvcol() rescanning the line after each insert. ... Indeed. Part of the problem is that most
        Message 3 of 17 , Jul 21, 2013
        • 0 Attachment
          On 20/07/2013 10:13, LCD 47 wrote:
          > On 19 July 2013, Mike Williams <mike.williams@...> wrote:
          >> On 19/07/2013 15:52, Mike Williams wrote:
          >>> On 19/07/2013 12:18, Dimitar DIMITROV wrote:
          >>>> Hi,
          >>>>
          >>>> Did a search on the vim_dev archives but couldn't find anything
          >>>> related to this. Sorry if this is redundant. Basically vim is
          >>>> exponentially slow after 1000000iHello <esc> as mentionned in this
          >>>> link:
          >>>> http://www.galexander.org/vim_sucks.html
          >>>
          >>> A quick run over with a profiler seems to show most of the time is
          >>> being spent in vim_strchr() called from has_format_option(), called
          >> >from internal_format(). If I first do :set paste then it is a lot
          >>> quicker - for 4000iHello<ESC> it goes from 16s down to 1s.
          > [...]
          >> Textwidth. Darn sight quicker with it set to 0, wish I had
          >> remembered that. Another factor of ~2 quicker for my final times,
          >> 16000iHello<ESC> now takes ~4s.
          > [...]
          >
          > Right, what you're seeing here is Vim backtracking to find a blank
          > to split that N * "Hello" line. Since there is no such blank, you get
          > len("Hello") * N * (N + 1) / 2 failed comparisons, which is exactly the
          > O(N^2) behaviour claimed in the article mentioned above.
          >
          > If you replace your test by, say, "16000iHello <ESC>" (with a space
          > after "Hello"), you get linear behaviour, namely len("Hello") * N failed
          > comparisons.
          >
          > I believe this can be fixed with a counter that means something
          > along the lines of: "this line is longer than &tw, and it has no
          > breaking point for the first X characters". Then X would be updated
          > every time more text is appended to that line.

          Yup to all that. And there is another O(n^2) operation is getvcol()
          rescanning the line after each insert.

          > On 19 July 2013, Marc Weber <marco-oweber@...> wrote:
          >> Let's discuss the use case, first. Is there one ?
          >
          > Yes, there is: you run through that piece of code every time you
          > press "p" to paste text. It hits you particularly hard when you edit
          > files with very long lines.
          >
          >> If not let's forgett about it ..
          > [...]
          >
          > Perhaps we should try to understand the problem before dismissing it
          > as harmless?

          Indeed. Part of the problem is that most computers these days have
          ridiculous amounts of processing power and storage that slow algorithms
          for small to medium amounts of data appear near instantaneous. On say
          an Atom or Arm based netbook then the problem becomes more apparent.
          Getting a faster machine is not always the solution.

          Mike
          --
          Born naked, wet and hungry... and it gets worse?

          --
          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_dev" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • Charles Campbell
          ... I tried this problem with both 1000000iHello 1000000iHello In both cases ctrl-c worked just fine to break the operation. I used vim 7.4a.35 on a
          Message 4 of 17 , Jul 22, 2013
          • 0 Attachment
            Dimitar DIMITROV wrote:
            > > > There is no use case
            > > If you do something stupid by accident most vim operations can be
            > > aborted by ctrl-c (exception: python, rbuy, .. scripts)
            >
            > Try to abort it you will see the success you have.
            >
            > > So there is still nothing to fix or talk about unless there is a use
            > > case.
            > >
            > > Marc Weber
            I tried this problem with both

            1000000iHello<esc>
            1000000iHello <esc>

            In both cases ctrl-c worked just fine to break the operation.

            I used vim 7.4a.35 on a linux system when trying this.

            Regards,
            C Campbell

            --
            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php

            ---
            You received this message because you are subscribed to the Google Groups "vim_dev" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
          • Nikolay Pavlov
            ... This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and
            Message 5 of 17 , Jul 22, 2013
            • 0 Attachment


              On Jul 20, 2013 2:28 AM, "Gary Johnson" <garyjohn@...> wrote:
              >
              > On 2013-07-19, Dimitar DIMITROV wrote:
              >
              > Hi Dimitar,
              >
              > It would be a big help to those of us with threading mail readers if
              > you would be sure that your replies include "Re: " at the start of
              > the Subject.

              This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and *this* field makes mail readers able to arrange messages correctly.

              > Thanks,
              > Gary
              >
              > --
              > --
              > You received this message from the "vim_dev" maillist.
              > Do not top-post! Type your reply below the text you are replying to.
              > For more information, visit http://www.vim.org/maillist.php
              >
              > ---
              > You received this message because you are subscribed to the Google Groups "vim_dev" group.
              > To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              > For more options, visit https://groups.google.com/groups/opt_out.
              >
              >

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php
               
              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
               
               
            • James McCoy
              ... That depends on the mail reader. Even with MUAs that follow the Message-ID/In-Reply-To chains, it s not uncommon to treat a topic change as a new thread.
              Message 6 of 17 , Jul 22, 2013
              • 0 Attachment
                On Tue, Jul 23, 2013 at 07:39:12AM +0400, Nikolay Pavlov wrote:
                > On Jul 20, 2013 2:28 AM, "Gary Johnson" <garyjohn@...> wrote:
                > >
                > > On 2013-07-19, Dimitar DIMITROV wrote:
                > >
                > > Hi Dimitar,
                > >
                > > It would be a big help to those of us with threading mail readers if
                > > you would be sure that your replies include "Re: " at the start of
                > > the Subject.
                >
                > This has nothing to do with threads. There is a special field added to
                > messages when you reply that contains unique ID of the message being
                > replied to and *this* field makes mail readers able to arrange messages
                > correctly.

                That depends on the mail reader. Even with MUAs that follow the
                Message-ID/In-Reply-To chains, it's not uncommon to treat a topic change
                as a new thread. For ones that don't honor the IDs, the subject is
                usually used instead.

                Regardless, Dimitar's emails don't even have the In-Reply-To header set.
                The X-Mailer header claims to be YahooMailWebService, so I doubt there's
                much that he can do to fix it short of deciding to use a more standard,
                standalone MUA instead of (I assume) their web interface.

                Cheers,
                --
                James
                GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy <jamessan@...>

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              • Gary Johnson
                ... I m well aware of those fields, and my mail reader (mutt) handles them quite well. It is also capable of threading messages that lack those fields and
                Message 7 of 17 , Jul 22, 2013
                • 0 Attachment
                  On 2013-07-23, Nikolay Pavlov wrote:
                  >
                  > On Jul 20, 2013 2:28 AM, "Gary Johnson" wrote:
                  > >
                  > > On 2013-07-19, Dimitar DIMITROV wrote:
                  > >
                  > > Hi Dimitar,
                  > >
                  > > It would be a big help to those of us with threading mail readers if
                  > > you would be sure that your replies include "Re: " at the start of
                  > > the Subject.
                  >
                  > This has nothing to do with threads. There is a special field added to messages
                  > when you reply that contains unique ID of the message being replied to and
                  > *this* field makes mail readers able to arrange messages correctly.

                  I'm well aware of those fields, and my mail reader (mutt) handles
                  them quite well. It is also capable of threading messages that lack
                  those fields and have only "Re: " in front of the original subject
                  to go by. Dimitar's messages did not contain those headers, though,
                  and since his Subject lines didn't even include the "Re: ", I
                  assumed it would be a stretch for his mail client to do any more
                  than allow him to add the "Re: ".

                  Regards,
                  Gary

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php

                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                • Nikolay Pavlov
                  ... messages ... and ... Good to know something new. One my mailer ignores everything but special header, another splits threads on theme change (I did not
                  Message 8 of 17 , Jul 23, 2013
                  • 0 Attachment


                    On Jul 23, 2013 9:24 AM, "Gary Johnson" <garyjohn@...> wrote:
                    >
                    > On 2013-07-23, Nikolay Pavlov wrote:
                    > >
                    > > On Jul 20, 2013 2:28 AM, "Gary Johnson" wrote:
                    > > >
                    > > > On 2013-07-19, Dimitar DIMITROV wrote:
                    > > >
                    > > > Hi Dimitar,
                    > > >
                    > > > It would be a big help to those of us with threading mail readers if
                    > > > you would be sure that your replies include "Re: " at the start of
                    > > > the Subject.
                    > >
                    > > This has nothing to do with threads. There is a special field added to messages
                    > > when you reply that contains unique ID of the message being replied to and
                    > > *this* field makes mail readers able to arrange messages correctly.
                    >
                    > I'm well aware of those fields, and my mail reader (mutt) handles
                    > them quite well.  It is also capable of threading messages that lack
                    > those fields and have only "Re: " in front of the original subject
                    > to go by.  Dimitar's messages did not contain those headers, though,
                    > and since his Subject lines didn't even include the "Re: ", I
                    > assumed it would be a stretch for his mail client to do any more
                    > than allow him to add the "Re: ".

                    Good to know something new. One my mailer ignores everything but special header, another splits threads on theme change (I did not realize I was getting separate threads for this very reason though), but does not join messages in threads if they lack field in question. So I assumed this field is required for all mailers without performing more research.

                    > Regards,
                    > Gary
                    >
                    > --
                    > --
                    > You received this message from the "vim_dev" maillist.
                    > Do not top-post! Type your reply below the text you are replying to.
                    > For more information, visit http://www.vim.org/maillist.php
                    >
                    > ---
                    > You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    > To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    > For more options, visit https://groups.google.com/groups/opt_out.
                    >
                    >

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php
                     
                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/groups/opt_out.
                     
                     
                  • Christian Brabandt
                    Hi Mike! ... Are you going to create a patch for that? ... Not only that, but getvcol() is called twice from the edit() function. Once from
                    Message 9 of 17 , Jul 30, 2013
                    • 0 Attachment
                      Hi Mike!

                      On So, 21 Jul 2013, Mike Williams wrote:

                      > On 20/07/2013 10:13, LCD 47 wrote:
                      > > I believe this can be fixed with a counter that means something
                      > >along the lines of: "this line is longer than &tw, and it has no
                      > >breaking point for the first X characters". Then X would be updated
                      > >every time more text is appended to that line.

                      Are you going to create a patch for that?

                      > Yup to all that. And there is another O(n^2) operation is getvcol()
                      > rescanning the line after each insert.

                      Not only that, but getvcol() is called twice from the edit() function.
                      Once from validate_cursor_col() at line 695 and once from
                      validate_cursor() at line 725. It seems like that either one of those
                      calls could be saved.

                      regards,
                      Christian
                      --
                      Wie man sein Kind nicht nennen sollte:
                      Hans Dampf

                      --
                      --
                      You received this message from the "vim_dev" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php

                      ---
                      You received this message because you are subscribed to the Google Groups "vim_dev" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                      For more options, visit https://groups.google.com/groups/opt_out.
                    Your message has been successfully submitted and would be delivered to recipients shortly.