Loading ...
Sorry, an error occurred while loading the content.

Re: Progress indicator for :TOhtml command

Expand Messages
  • Yongwei Wu
    ... I tried it on Windows, and the display was too flashy and intrusive. I can t say I like it. And there was nothing to fix about the tabs. They were
    Message 1 of 28 , May 31, 2010
    • 0 Attachment
      On 1 June 2010 04:47, Benjamin Fritz <fritzophrenic@...> wrote:
      > I love using the :TOhtml command, and I keep finding more ways to use
      > it. Recently, I had a large-ish log file (several thousand lines), in
      > which I wanted to call attention to a few groups of lines, but I
      > figured people may want the context as well. So, I set up some folds
      > and some quick syntax highlighting, and went to go create an html copy
      > of it using the "dynamic folding" feature of the command.
      >
      > Unfortunately, I discovered that processing such a large file, even
      > with no syntax highlighting, takes a *very* long time. I probably
      > should have selected just a smaller area of interest but...
      >
      > I waited quite a while, and finally hit CTRL-C to stop it. Luckily it
      > hadn't actually gotten that far (probably about 30%), but I was
      > worried that it may have been almost done, and all I needed to do was
      > wait a bit longer.
      >
      > So anyway, for future use, I wanted to be able to see quickly whether
      > the conversion was worth waiting for. Therefore, I have written a
      > patch to add a progress indicator. When I opened the file, I noticed
      > that indentation is quite a mess (a mix of tabs and spaces, sometimes
      > in the same line), so I also fixed that up (by using gg=G with noet,
      > sw=2, sts=2, and ts=2). For clarity, I ran the diff on the file
      > *after* fixing the ident.
      >
      > Since the indentation patch is actually *larger* than the file itself,
      > I have just attached the fully patched file instead of a patch. This
      > will also make it easier for Windows users to try :-)
      >
      > Please comment...especially if you know of a better way to accomplish
      > something similar. I tried using :echon to draw the progress bar, but
      > the echo was being cleared each time through the main loop, so I
      > switched to using :echo with a string that is gradually built as the
      > script processes.

      I tried it on Windows, and the display was too flashy and intrusive.
      I can't say I like it.

      And there was nothing to "fix" about the tabs. They were perfectly
      fine. Tabstop was set to the default (8), and only shiftwidth was set
      to 2 (along with sts, which does not affect the saved file). You
      seemed to have confused them. The original version is easier to view
      (using type/cat/more/less or nearly anything).

      --
      Wu Yongwei
      URL: http://wyw.dcweb.cn/

      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • Christian J. Robinson
      ... I tried it too--on Windows and on Linux--and it was flashy and unusable on both, plus about 15 to 20 percent slower than the original on both. I would be
      Message 2 of 28 , May 31, 2010
      • 0 Attachment
        On Tue, 1 Jun 2010, Yongwei Wu wrote:

        > I tried it on Windows, and the display was too flashy and intrusive.
        > I can't say I like it.

        I tried it too--on Windows and on Linux--and it was flashy and
        unusable on both, plus about 15 to 20 percent slower than the original
        on both.

        I would be for having a progress meter, if it could be implemented in
        a better way without slowing the script down.

        My best suggestion is just to output one line every 5 to 10 percent
        through processing with the current progress, e.g.:

        0%
        5%
        10%
        15%
        [...]
        95%
        Done

        - Christian

        --
        Abdominos... sit-ups & pizza.
        Christian J. Robinson <heptite@...> http://christianrobinson.name/

        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • Andy Wokula
        ... I d suggest to try one of the existing progress bars (sorry, didn t try them myself yet):
        Message 3 of 28 , Jun 1, 2010
        • 0 Attachment
          Am 31.05.2010 22:47, schrieb Benjamin Fritz:
          > I love using the :TOhtml command, and I keep finding more ways to use
          > it. Recently, I had a large-ish log file (several thousand lines), in
          > which I wanted to call attention to a few groups of lines, but I
          > figured people may want the context as well. So, I set up some folds
          > and some quick syntax highlighting, and went to go create an html copy
          > of it using the "dynamic folding" feature of the command.
          >
          > Unfortunately, I discovered that processing such a large file, even
          > with no syntax highlighting, takes a *very* long time. I probably
          > should have selected just a smaller area of interest but...
          >
          > I waited quite a while, and finally hit CTRL-C to stop it. Luckily it
          > hadn't actually gotten that far (probably about 30%), but I was
          > worried that it may have been almost done, and all I needed to do was
          > wait a bit longer.
          >
          > So anyway, for future use, I wanted to be able to see quickly whether
          > the conversion was worth waiting for. Therefore, I have written a
          > patch to add a progress indicator.

          I'd suggest to try one of the existing progress bars (sorry, didn't try
          them myself yet):

          http://github.com/tomtom/vimtlib/blob/master/autoload/tlib/progressbar.vim
          http://www.vim.org/scripts/script.php?script_id=2006

          --
          Andy

          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        • Ben Fritz
          ... I am fine with tabs or spaces for indentation. I am NOT fine with a mixture of the two. From the previous modeline, I assumed that the intent was a
          Message 4 of 28 , Jun 1, 2010
          • 0 Attachment
            On May 31, 10:50 pm, Yongwei Wu <wuyong...@...> wrote:
            > On 1 June 2010 04:47, Benjamin Fritz <fritzophre...@...> wrote:
            >
            >
            > And there was nothing to "fix" about the tabs.  They were perfectly
            > fine.  Tabstop was set to the default (8), and only shiftwidth was set
            > to 2 (along with sts, which does not affect the saved file).  You
            > seemed to have confused them.  The original version is easier to view
            > (using type/cat/more/less or nearly anything).
            >

            I am fine with tabs or spaces for indentation. I am NOT fine with a
            mixture of the two. From the previous modeline, I assumed that the
            intent was a tab-based indent. You are free to change the tabstop to
            view the file, but if you edit the file using a 'shiftwidth' and
            'tabstop' value that differ, the indentation will become a mixture of
            tabs and spaces, so that changing tabstop no longer looks right.

            I did not modify the value of 'sts', that was already there.

            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
          • Ben Fritz
            On May 31, 11:11 pm, Christian J. Robinson ... On May 31, 11:11 pm, Christian J. Robinson ... Hmm, strange. I did
            Message 5 of 28 , Jun 1, 2010
            • 0 Attachment
              On May 31, 11:11 pm, "Christian J. Robinson" <hept...@...>
              wrote:
              > On Tue, 1 Jun 2010, Yongwei Wu wrote:
              > > I tried it on Windows, and the display was too flashy and intrusive.
              > > I can't say I like it.


              On May 31, 11:11 pm, "Christian J. Robinson" <hept...@...>
              wrote:
              > I would be for having a progress meter, if it could be implemented in
              > a better way without slowing the script down.
              >
              > My best suggestion is just to output one line every 5 to 10 percent
              > through processing with the current progress, e.g.:
              >
              >   0%
              >   5%
              >   10%
              >   15%
              >   [...]
              >   95%
              >   Done
              >

              Hmm, strange. I did not notice a slow-down from the original script,
              but then I also set 'nomore' and fdm=manual as a way to speed things
              up a little in addition to the progress bar changes. On my dinosaur of
              a computer, processing 2html.vim took about 40 seconds LESS with the
              patched version that with the unpatched version, according to the
              before/after times returned by localtime(). What did you do to see the
              slowdown?

              I also dislike the flashiness, but couldn't find a way around it. The
              echo'd text gets cleared every pass through the main processing loop.
              For this reason, a periodic (5, 10, 20%, etc.) message wasn't going to
              work. I would certainly prefer this method...my first attempt used
              echon to display one "tick" every few lines processed (a tick being
              calculated to almost fill the current &columns setting). Does someone
              have an idea of how I might accomplish this?

              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php
            • Ben Fritz
              ... Hmm, interesting...and thanks! These two plugins seem to use the statusline for the progress indicator. I thought of that, and abandoned the idea when I
              Message 6 of 28 , Jun 1, 2010
              • 0 Attachment
                On Jun 1, 7:21 am, Andy Wokula <anw...@...> wrote:
                >
                > I'd suggest to try one of the existing progress bars (sorry,  didn't try
                > them myself yet):
                >
                > http://github.com/tomtom/vimtlib/blob/master/autoload/tlib/progressba...http://www.vim.org/scripts/script.php?script_id=2006
                >

                Hmm, interesting...and thanks! These two plugins seem to use the
                statusline for the progress indicator. I thought of that, and
                abandoned the idea when I noticed that the status lines weren't being
                populated all the way during processing...but later investigation
                showed me that's because 'ruler' gets reset from one of the speed-up
                options set by the plugin (I don't remember which). At the time, I was
                still hoping to use :echon to output the progress indicator, which I
                also had to abandon. My eventual goal is to use the full screen width
                (or pretty close to it) for the progress bar. Is there a good way to
                get this for an individual status line? I didn't think so, whereas
                using an ":echo" type command, &columns basically does the trick.

                I'll give it a shot using the status line (maybe making use of a
                plugin, but I'm hoping for eventual inclusion into the upstream, so
                maybe just pulling code from them). Hopefully it will be more usable.
                For me, the screen flashes quite a bit *without* the progress
                indicator, so I'm not certain it will be, but perhaps the somewhat
                static status line will be more readable than a quickly flashing
                message at the bottom of the window.

                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php
              • Ingo Karkat
                ... Personal preferences aside, the syntax/2html.vim looks like a correct softtabstop=2 to me; I use this setting all the time, and Vim supports it well. (Cp.
                Message 7 of 28 , Jun 1, 2010
                • 0 Attachment
                  On 01-Jun-2010 16:43, Ben Fritz wrote:
                  > I am fine with tabs or spaces for indentation. I am NOT fine with a
                  > mixture of the two. From the previous modeline, I assumed that the
                  > intent was a tab-based indent. You are free to change the tabstop to
                  > view the file, but if you edit the file using a 'shiftwidth' and
                  > 'tabstop' value that differ, the indentation will become a mixture of
                  > tabs and spaces, so that changing tabstop no longer looks right.

                  Personal preferences aside, the syntax/2html.vim looks like a correct
                  softtabstop=2 to me; I use this setting all the time, and Vim supports it well.
                  (Cp. :help 'softtabstop'). The only downside is that you should keep ts=8, and
                  not mess with 'shiftwidth' and related settings while editing. In that regard,
                  it's less flexible than a pure tab or space-indented format, you're right.

                  -- regards, ingo

                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php
                • Christian J. Robinson
                  ... Time elapsed about 39 seconds. ... Time elapsed about 31 seconds. So the first is over 20% slower. This morning I tried it under a vim -u NONE --noplugin
                  Message 8 of 28 , Jun 1, 2010
                  • 0 Attachment
                    On Tue, 1 Jun 2010, Ben Fritz wrote:

                    > What did you do to see the slowdown?

                    In two separate Vim instances ("vim 2html.vim"--not at the same time):

                    :let timestart=join(reltime(), ' ') | exe 'so ' . expand('%') | let timefinish=join(reltime(), ' ')
                    :echo timestart | echo timefinish

                    Time elapsed about 39 seconds.

                    :let timestart=join(reltime(), ' ') | exe 'TOhtml' | let timefinish=join(reltime(), ' ')
                    :echo timestart | echo timefinish

                    Time elapsed about 31 seconds.

                    So the first is over 20% slower.

                    This morning I tried it under a "vim -u NONE --noplugin 2html.vim"
                    instance and your version was actually four seconds faster than the
                    original. Odd.

                    I do have a fairly complex vimrc, as well as a number of plugins
                    installed.

                    - Christian

                    --
                    Christian J. Robinson <heptite@...>

                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php
                  • Christian Brabandt
                    ... Here is an experimental version, in which I included the essential part from the mentioned vim.org/script into the script from Ben. Bugs: - currently only
                    Message 9 of 28 , Jun 1, 2010
                    • 0 Attachment
                      On Tue, June 1, 2010 2:21 pm, Andy Wokula wrote:
                      > I'd suggest to try one of the existing progress bars (sorry, didn't try
                      > them myself yet):
                      >
                      > http://github.com/tomtom/vimtlib/blob/master/autoload/tlib/progressbar.vim
                      > http://www.vim.org/scripts/script.php?script_id=2006

                      Here is an experimental version, in which I included the essential part
                      from the mentioned vim.org/script into the script from Ben.

                      Bugs: - currently only works well with a big enough window.
                      - progressbar requires redrawing the window, while processing
                      the 2html.vim script. This means, it slows it down.
                      - requires a +float (no check yet)

                      regards,
                      Christian

                      --
                      You received this message from the "vim_dev" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php
                    • Ben Fritz
                      I ve done some updates...more to follow. I forgot to hit the email updates to me button on the google groups interface so that I can upload attachments on a
                      Message 10 of 28 , Jun 1, 2010
                      • 0 Attachment
                        I've done some updates...more to follow. I forgot to hit the "email
                        updates to me" button on the google groups interface so that I can
                        upload attachments on a reply.

                        --
                        You received this message from the "vim_dev" maillist.
                        Do not top-post! Type your reply below the text you are replying to.
                        For more information, visit http://www.vim.org/maillist.php
                      • Benjamin Fritz
                        ... I like this look a lot better! I made a few improvements: - Change the color of StatusLineNC to match StatusLine (and restore it at the end) to reduce
                        Message 11 of 28 , Jun 1, 2010
                        • 0 Attachment
                          >
                          > Here is an experimental version, in which I included the essential part
                          > from the mentioned vim.org/script into the script from Ben.
                          >
                          > Bugs: - currently only works well with a big enough window.
                          > - progressbar requires redrawing the window, while processing
                          > the 2html.vim script. This means, it slows it down.
                          > - requires a +float (no check yet)
                          >

                          I like this look a lot better! I made a few improvements:

                          - Change the color of StatusLineNC to match StatusLine (and restore it at the
                          end) to reduce flashing of the statusline.
                          - Draw the progress bar in the new window instead of the original window, to
                          reduce the amout of time from the screen clear to the redraw when redrawing
                          the windows, reducing the flashing effect.
                          - Set the size of the new window to only 2 lines, so that only 1 line of the
                          buffer actually appears. The intent of this was to further reduce the flashing
                          effect, but it also has the nice side affect of making the conversion MUCH
                          faster, since less of the expensive html syntax highlighting needs to be
                          completed every redraw.
                          - Added a second progress bar for the attributes processing (previously I was
                          doing this with a %d/%d printf).
                          - Fixed bug in progress bar incr function that prevented incrementing by large
                          amounts, that are still less than the maximum value.
                          - Fixed bug where progress bar would not fill all the way when doing non-dynamic
                          folds.
                          - Fixed off-by-one error in progress bar initialization.

                          I also removed the "redrawstatus" commands. They did not seem to be necessary,
                          and only served to increase the processing time. Did you have a particular
                          reason to include them?

                          --
                          You received this message from the "vim_dev" maillist.
                          Do not top-post! Type your reply below the text you are replying to.
                          For more information, visit http://www.vim.org/maillist.php
                        • Benjamin Fritz
                          ... And one more bugfix, to prevent throwing errors in the cleanup commands at the end. With the attached script on fairly recent hardware, I get the following
                          Message 12 of 28 , Jun 1, 2010
                          • 0 Attachment
                            On Tue, Jun 1, 2010 at 6:04 PM, Benjamin Fritz <fritzophrenic@...> wrote:
                            >
                            > I like this look a lot better! I made a few improvements:
                            >

                            And one more bugfix, to prevent throwing errors in the cleanup
                            commands at the end.

                            With the attached script on fairly recent hardware, I get the
                            following timings (in seconds) when converting the 2html.vim script to
                            html, several times and discarding the first few trials (these were
                            outliers...around 100 seconds, probably while getting the cache all
                            ironed out or something):

                            with progress bar:
                            34
                            34
                            35
                            32
                            32
                            without progress bar (using let html_no_progress=1):
                            33
                            33
                            33
                            33
                            32
                            32
                            33
                            30
                            unmodified "official" script (with autocmds to set winsize and
                            foldmethod to prevent syntax highlighting to screw with the results):
                            31
                            32
                            31
                            31

                            For me anyway, the difference made by the progress bar, with the
                            optimizations in the attached version of 2html.vim, is negligible.

                            So...is the screen flashing still too distracting? I think the version
                            attached handles it well enough, at least for my purposes.

                            Any additional comments or fixes?

                            --
                            You received this message from the "vim_dev" maillist.
                            Do not top-post! Type your reply below the text you are replying to.
                            For more information, visit http://www.vim.org/maillist.php
                          • Ben Fritz
                            ... Interesting. Is this true, even with the progress bar on the top window as in the latest version? If it is on the bottom, the screen clears again for a
                            Message 13 of 28 , Jun 2, 2010
                            • 0 Attachment
                              On Jun 2, 3:13 am, "Christian Brabandt" <cbli...@...> wrote:
                              > On Wed, June 2, 2010 1:04 am, Benjamin Fritz wrote:
                              > > - Added a second progress bar for the attributes processing (previously I
                              > > was
                              > >   doing this with a %d/%d printf).
                              > > I also removed the "redrawstatus" commands. They did not seem to be
                              > > necessary,
                              > > and only served to increase the processing time. Did you have a particular
                              > > reason to include them?
                              >
                              > Yes. I can't see the progressbar otherwise. It is not updating for me. So
                              > I suggest, enabling it again.
                              >

                              Interesting. Is this true, even with the progress bar on the top
                              window as in the latest version? If it is on the bottom, the screen
                              clears again for a redraw immediately after the bottom status line is
                              drawn, making it very hard to see the progress bar...which is why I
                              put it on the top.

                              I see from :help :redrawstatus,

                              Useful to update the status line(s) when 'statusline'
                              includes an item that doesn't cause automatic
                              updating.

                              But we're setting the statusline to a new (static) value...surely the
                              statusline is redrawn when 'statusline' is set to something new?

                              If we leave the progress bar on the top window (which I like...any
                              disagreements?) we need to find the appropriate place to put
                              the :redrawstatus command, so that we redraw the correct window's
                              status line. I don't think it's a great idea to use redrawstatus!,
                              although a "normal" use case is probably just the two windows, so
                              maybe it's not that much more expensive?

                              I toyed with the idea of only updating the 'statusline' option and
                              calling :redrawstatus if the progress bar has changed position in a
                              visible way, but on my system (Windows XP, gvim 7.2.437) this wasn't
                              needed.

                              On an unrelated note...should we continue this on vim_use, vim_dev, or
                              both? I original posted to both groups so that I could potentially get
                              more comments, but now it seems we're deep in developer mode.

                              --
                              You received this message from the "vim_dev" maillist.
                              Do not top-post! Type your reply below the text you are replying to.
                              For more information, visit http://www.vim.org/maillist.php
                            • Ben Fritz
                              ... This got me thinking...so tried the following: gvim -N -u NONE -i NONE filetype indent plugin on syntax on ... I do not see the progress bars! The status
                              Message 14 of 28 , Jun 2, 2010
                              • 0 Attachment
                                On Jun 2, 8:43 am, Ben Fritz <fritzophre...@...> wrote:
                                > On Jun 2, 3:13 am, "Christian Brabandt" <cbli...@...> wrote:
                                >
                                > > On Wed, June 2, 2010 1:04 am, Benjamin Fritz wrote:
                                > > > - Added a second progress bar for the attributes processing (previously I
                                > > > was
                                > > >   doing this with a %d/%d printf).
                                > > > I also removed the "redrawstatus" commands. They did not seem to be
                                > > > necessary,
                                > > > and only served to increase the processing time. Did you have a particular
                                > > > reason to include them?
                                >
                                > > Yes. I can't see the progressbar otherwise. It is not updating for me. So
                                > > I suggest, enabling it again.
                                >
                                > Interesting. Is this true, even with the progress bar on the top
                                > window as in the latest version? If it is on the bottom, the screen
                                > clears again for a redraw immediately after the bottom status line is
                                > drawn, making it very hard to see the progress bar...which is why I
                                > put it on the top.
                                >

                                This got me thinking...so tried the following:

                                gvim -N -u NONE -i NONE
                                filetype indent plugin on
                                syntax on
                                :help intro
                                :source $HOME/vimfiles/syntax/2html.vim

                                I do not see the progress bars! The status lines never get drawn at
                                all (and the conversion is lightning fast). It appears I have
                                something in my Vim setup that slows things down enough, or redraws
                                the window, or something, that causes the status lines to appear when
                                normally they would not.

                                Any ideas? I don't see anything suspicious in my usual Buf/Win Enter/
                                Leave autocmds:


                                --- Auto-Commands ---
                                style_highlight WinEnter
                                * if !exists('w:created') | call
                                matchadd('WhitespaceError','\S\@<=\s\+\%#\@<!$') | endif
                                if &filetype=~'\v<%(c|vim|dosbatch)>' && !
                                exists('w:tabs_are_bad') | let w:tabs_are_bad =
                                matchadd('WhitespaceError',"\t") | endif
                                misc WinEnter
                                * if !exists('w:created') && &ft!='qf' | setlocal nowrap
                                | endif
                                matchparen WinEnter
                                * call s:Highlight_Matching_Pair()
                                misc WinEnter
                                * let w:created=1

                                --- Auto-Commands ---
                                insertmode WinLeave
                                * if exists('w:last_fdm') | let &l:foldmethod=w:last_fdm |
                                unlet w:last_fdm | endif

                                --- Auto-Commands ---
                                filetypedetect BufEnter
                                *.xpm if getline(1) =~ "XPM2" | setf xpm2 | else | setf
                                xpm | endif
                                *.xpm2 setf xpm2
                                misc BufEnter
                                * if (&ft=='qf' || &previewwindow || bufname('%') ==#
                                "__Tag_List__") && !exists('s:scrolloff_sav') | let
                                s:scrolloff_sav=&scrolloff | set scrolloff=0 | endif
                                repeatPlugin BufEnter
                                * if g:repeat_tick == 0|let g:repeat_tick = b:changedtick|
                                endif
                                FileExplorer BufEnter
                                * silent! call s:LocalBrowse(expand("<amatch>"))
                                .* silent! call s:LocalBrowse(expand("<amatch>"))
                                BufEnter
                                *.vba setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
                                'unix'| setlocal ma ff=unix noma |endif|call
                                vimball#ShowMesg(0,"Source this file to extract it! (:so %)")
                                *.vba.gz setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
                                'unix'| setlocal ma ff=unix noma |endif|call
                                vimball#ShowMesg(0,"Source this file to extract it! (:so %)")
                                *.vba.bz2 setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
                                'unix'| setlocal ma ff=unix noma |endif|call
                                vimball#ShowMesg(0,"Source this file to extract it! (:so %)")
                                *.vba.zip setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
                                'unix'| setlocal ma ff=unix noma |endif|call
                                vimball#ShowMesg(0,"Source this file to extract it! (:so %)")

                                --- Auto-Commands ---
                                misc BufLeave
                                * if (&ft=='qf' || &previewwindow || bufname('%') ==#
                                "__Tag_List__") && exists('s:scrolloff_sav') | let
                                &scrolloff=s:scrolloff_sav | unlet s:scrolloff_sav | endif
                                repeatPlugin BufLeave
                                * let g:repeat_tick = (g:repeat_tick == b:changedtick ||
                                g:repeat_tick == 0) ? 0 : -1

                                --
                                You received this message from the "vim_dev" maillist.
                                Do not top-post! Type your reply below the text you are replying to.
                                For more information, visit http://www.vim.org/maillist.php
                              • Benjamin Fritz
                                ... Found it. I have the following in my .vimrc: Highlight trailing whitespace, unless it is being entered now autocmd WinEnter,VimEnter * if
                                Message 15 of 28 , Jun 2, 2010
                                • 0 Attachment
                                  On Wed, Jun 2, 2010 at 8:55 AM, Ben Fritz <fritzophrenic@...> wrote:
                                  > It appears I have
                                  > something in my Vim setup that slows things down enough, or redraws
                                  > the window, or something, that causes the status lines to appear when
                                  > normally they would not.
                                  >
                                  > Any ideas? I don't see anything suspicious in my usual Buf/Win Enter/
                                  > Leave autocmds:
                                  >

                                  Found it. I have the following in my .vimrc:

                                  " Highlight trailing whitespace, unless it is being entered now
                                  autocmd WinEnter,VimEnter *
                                  \ if !exists('w:created') |
                                  \ call matchadd('WhitespaceError','\S\@<=\s\+\%#\@<!$') |
                                  \ endif
                                  " necessary to have it highlight a just-entered trailing space
                                  autocmd InsertLeave * redraw!

                                  The InsertLeave command is triggering because 2html works using
                                  normal! a... commands. I have the autocmd because of the following
                                  text in :help /\%# :

                                  WARNING: When the cursor is moved after the pattern was used, the
                                  result becomes invalid. Vim doesn't automatically update the matches.
                                  This is especially relevant for syntax highlighting and 'hlsearch'.
                                  In other words: When the cursor moves the display isn't updated for
                                  this change. An update is done for lines which are changed (the whole
                                  line is updated) or when using the |CTRL-L| command (the whole screen
                                  is updated).

                                  So anyway, it looks like we need to do the redrawstatus. It would be
                                  easiest to just use redrawstatus! right after calling the incr
                                  function. I wonder how much of a performance impact this has? If it is
                                  significant, we should probably call it only sometimes, perhaps after
                                  a progress bar position update (which would mean the number of
                                  processed lines would not be updated in real-time on large files). Any
                                  thoughts/input?

                                  It appears we also need to ignore some events; at least BufEnter,
                                  WinEnter, InsertEnter, BufLeave, WinLeave, and InsertLeave. Syntax
                                  seems like a good idea as well, but then we'd want to do the syntax
                                  highlight when done, so perhaps we can leave this one in. Thoughts on
                                  this? Is this really needed as part of the same patch or would this be
                                  a future, performance-enhancing-only, patch?

                                  --
                                  You received this message from the "vim_dev" maillist.
                                  Do not top-post! Type your reply below the text you are replying to.
                                  For more information, visit http://www.vim.org/maillist.php
                                • Christian Brabandt
                                  ... Try the attached version: - Check for +float - Should work better with smaller window sizes - Make the progressbar for the attribute processing slightly
                                  Message 16 of 28 , Jun 3, 2010
                                  • 0 Attachment
                                    On Wed, June 2, 2010 4:36 am, Benjamin Fritz wrote:

                                    > Any additional comments or fixes?

                                    Try the attached version:

                                    - Check for +float
                                    - Should work better with smaller window sizes
                                    - Make the progressbar for the attribute processing slightly slower
                                    (it was too fast, to notice it)
                                    - small enhancements to how the progressbar works and how it displays.
                                    - don't show any content from the html window

                                    regards,
                                    Christian

                                    --
                                    You received this message from the "vim_dev" maillist.
                                    Do not top-post! Type your reply below the text you are replying to.
                                    For more information, visit http://www.vim.org/maillist.php
                                  • Benjamin Fritz
                                    ... More tweaks. This one is about twice as fast as Christian s, which it accomplishes by only redrawing when the progress bar has changed position. Question:
                                    Message 17 of 28 , Jun 3, 2010
                                    • 0 Attachment
                                      On Thu, Jun 3, 2010 at 3:18 PM, Christian Brabandt <cblists@...> wrote:
                                      >
                                      > Try the attached version:
                                      >
                                      > - Check for +float
                                      > - Should work better with smaller window sizes
                                      > - Make the progressbar for the attribute processing slightly slower
                                      >  (it was too fast, to notice it)
                                      > - small enhancements to how the progressbar works and how it displays.
                                      > - don't show any content from the html window
                                      >

                                      More tweaks. This one is about twice as fast as Christian's, which it
                                      accomplishes by only redrawing when the progress bar has changed
                                      position.

                                      Question: Christian's version calls :redrawstatus on the original
                                      window, but the new window is updated perfectly fine. :help
                                      :redrawstatus seems to indicate that only the current window will be
                                      redrawn unless the ! is given. What gives?

                                      Regardless, I have fixed the above issue and made a couple more minor
                                      fixes, including getting the entire title to display on my teensy
                                      laptop screen.

                                      This version is still not fast enough though. It is about 30% slower
                                      when the progress bar is enabled than when it is disabled. While I
                                      consider it a good tradeoff in most cases, we could certainly make it
                                      better.

                                      It would probably be faster to pre-calculate the line numbers needed
                                      to advance the progress bar rather than doing a bunch of
                                      floating-point math every cycle.

                                      --
                                      You received this message from the "vim_dev" maillist.
                                      Do not top-post! Type your reply below the text you are replying to.
                                      For more information, visit http://www.vim.org/maillist.php
                                    • Benjamin Fritz
                                      ... I ve attached a new version which pre-calculates the (integer) line numbers needed to advance the progress bar. Now all the floating point math is done
                                      Message 18 of 28 , Jun 5, 2010
                                      • 0 Attachment
                                        On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <fritzophrenic@...> wrote:
                                        > This version is still not fast enough though. It is about 30% slower
                                        > when the progress bar is enabled than when it is disabled. While I
                                        > consider it a good tradeoff in most cases, we could certainly make it
                                        > better.
                                        >
                                        > It would probably be faster to pre-calculate the line numbers needed
                                        > to advance the progress bar rather than doing a bunch of
                                        > floating-point math every cycle.
                                        >

                                        I've attached a new version which pre-calculates the (integer) line
                                        numbers needed to advance the progress bar. Now all the floating point
                                        math is done once, up front.

                                        The difference is not really very perceptible. I timed the execution
                                        on two files. First, I did the 5148-line autoload/phpcomplete.vim
                                        script. Timings were as follows on my laptop:

                                        progress disabled:
                                        average: 46 seconds

                                        floating-point progress:
                                        average: 61 seconds
                                        slowdown: 15 seconds longer than without progress bar
                                        percentage: 33% longer than without progress bar

                                        precalculated progress:
                                        average: 62 seconds
                                        slowdown: 16 seconds longer than without progress bar
                                        percentage: 35% longer than without progress bar

                                        Next I did a 33258-line C code file:

                                        progress disabled:
                                        average: 691 seconds

                                        floating-point progress:
                                        average: 716 seconds
                                        slowdown: 25 seconds longer than without progress bar
                                        percentage: 4% longer than without progress bar

                                        precalculated progress:
                                        average: 711 seconds
                                        slowdown: 20 seconds longer than without progress bar
                                        percentage: 3% longer than without progress bar

                                        I also did a number of very small sections of files (my usual use case
                                        for 2html) and did not notice any significant slowing; it only takes
                                        1-2 seconds longer for a 100 or 200 line selection.

                                        I take a few things from this.

                                        First of all, I don't think we'll get much performance improvement
                                        with this method. I do not know whether it is setting the status line
                                        and redrawing it, or whether it is the use of the object-oriented
                                        style functions, but it would probably require a different approach to
                                        get a significant speedup. I certainly like the look a lot better than
                                        the echo method, even if we could get echon working. Is a 10-20 second
                                        slow-down acceptable on large numbers of lines, if the normal
                                        execution time is measured in minutes anyway? To me, it certainly is.
                                        If something is going to be taking more than a few minutes, I want a
                                        progress bar to tell me whether it's worth letting it continue. Since
                                        the slow-down can be significant for midsize files, we will certainly
                                        need to mention in the :help that disabling the progress bar will make
                                        the conversion faster. Maybe we should only show the progress bar
                                        after some amount of time has elapsed? We could suppress the
                                        redrawstatus until 10 seconds have passed, or something like that. Any
                                        thoughts?

                                        Secondly, the precalculated version is not really any faster than the
                                        full floating-point calculation every cycle. I don't really have an
                                        opinion of which method gives more readable code. Does anyone else
                                        have any opinions on which version to keep? I think it would be
                                        possible to do away with floating point calculations entirely using
                                        the precalculated version; currently floating point is only used in
                                        the calculate_ticks function. This might be desireable so that we can
                                        remove the dependence on the +float feature, which is not marked with
                                        a "smallest version" indicator in :help +feature-list. This apparently
                                        means it is "system dependent". Does this mean float is pretty much
                                        always included, unless it is explicitly removed? How common are Vims
                                        without floating-point support? I already added use of the split()
                                        function, which was added in version 7, so this won't work on really
                                        old Vims...but do we want to support Vim 7.1 and earlier?

                                        --
                                        You received this message from the "vim_dev" maillist.
                                        Do not top-post! Type your reply below the text you are replying to.
                                        For more information, visit http://www.vim.org/maillist.php
                                      • ZyX
                                        Ответ на сообщение «Re: Progress indicator for :TOhtml command», присланное в 19:39:15 05 июня 2010, Суббота,
                                        Message 19 of 28 , Jun 5, 2010
                                        • 0 Attachment
                                          Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
                                          присланное в 19:39:15 05 июня 2010, Суббота,
                                          отправитель Benjamin Fritz:

                                          Why do you need float math for progress bar? Integer division is enough unless
                                          you are going to show progress in %.1f or even more precious format. %3.d format
                                          that you are using is integer and you obviously can't show half of a character.

                                          Текст сообщения:
                                          > On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <fritzophrenic@...>
                                          wrote:
                                          > > This version is still not fast enough though. It is about 30% slower
                                          > > when the progress bar is enabled than when it is disabled. While I
                                          > > consider it a good tradeoff in most cases, we could certainly make it
                                          > > better.
                                          > >
                                          > > It would probably be faster to pre-calculate the line numbers needed
                                          > > to advance the progress bar rather than doing a bunch of
                                          > > floating-point math every cycle.
                                          >
                                          > I've attached a new version which pre-calculates the (integer) line
                                          > numbers needed to advance the progress bar. Now all the floating point
                                          > math is done once, up front.
                                          >
                                          > The difference is not really very perceptible. I timed the execution
                                          > on two files. First, I did the 5148-line autoload/phpcomplete.vim
                                          > script. Timings were as follows on my laptop:
                                          >
                                          > progress disabled:
                                          > average: 46 seconds
                                          >
                                          > floating-point progress:
                                          > average: 61 seconds
                                          > slowdown: 15 seconds longer than without progress bar
                                          > percentage: 33% longer than without progress bar
                                          >
                                          > precalculated progress:
                                          > average: 62 seconds
                                          > slowdown: 16 seconds longer than without progress bar
                                          > percentage: 35% longer than without progress bar
                                          >
                                          > Next I did a 33258-line C code file:
                                          >
                                          > progress disabled:
                                          > average: 691 seconds
                                          >
                                          > floating-point progress:
                                          > average: 716 seconds
                                          > slowdown: 25 seconds longer than without progress bar
                                          > percentage: 4% longer than without progress bar
                                          >
                                          > precalculated progress:
                                          > average: 711 seconds
                                          > slowdown: 20 seconds longer than without progress bar
                                          > percentage: 3% longer than without progress bar
                                          >
                                          > I also did a number of very small sections of files (my usual use case
                                          > for 2html) and did not notice any significant slowing; it only takes
                                          > 1-2 seconds longer for a 100 or 200 line selection.
                                          >
                                          > I take a few things from this.
                                          >
                                          > First of all, I don't think we'll get much performance improvement
                                          > with this method. I do not know whether it is setting the status line
                                          > and redrawing it, or whether it is the use of the object-oriented
                                          > style functions, but it would probably require a different approach to
                                          > get a significant speedup. I certainly like the look a lot better than
                                          > the echo method, even if we could get echon working. Is a 10-20 second
                                          > slow-down acceptable on large numbers of lines, if the normal
                                          > execution time is measured in minutes anyway? To me, it certainly is.
                                          > If something is going to be taking more than a few minutes, I want a
                                          > progress bar to tell me whether it's worth letting it continue. Since
                                          > the slow-down can be significant for midsize files, we will certainly
                                          > need to mention in the :help that disabling the progress bar will make
                                          > the conversion faster. Maybe we should only show the progress bar
                                          > after some amount of time has elapsed? We could suppress the
                                          > redrawstatus until 10 seconds have passed, or something like that. Any
                                          > thoughts?
                                          >
                                          > Secondly, the precalculated version is not really any faster than the
                                          > full floating-point calculation every cycle. I don't really have an
                                          > opinion of which method gives more readable code. Does anyone else
                                          > have any opinions on which version to keep? I think it would be
                                          > possible to do away with floating point calculations entirely using
                                          > the precalculated version; currently floating point is only used in
                                          > the calculate_ticks function. This might be desireable so that we can
                                          > remove the dependence on the +float feature, which is not marked with
                                          > a "smallest version" indicator in :help +feature-list. This apparently
                                          > means it is "system dependent". Does this mean float is pretty much
                                          > always included, unless it is explicitly removed? How common are Vims
                                          > without floating-point support? I already added use of the split()
                                          > function, which was added in version 7, so this won't work on really
                                          > old Vims...but do we want to support Vim 7.1 and earlier?
                                          >
                                        • ZyX
                                          Ответ на сообщение «Re: Progress indicator for :TOhtml command», присланное в 19:39:15 05 июня 2010, Суббота,
                                          Message 20 of 28 , Jun 5, 2010
                                          • 0 Attachment
                                            Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
                                            присланное в 19:39:15 05 июня 2010, Суббота,
                                            отправитель Benjamin Fritz:

                                            A small benchmark for your and mine scripts:
                                            user system cpu total Relative
                                            mine, no progress 93,07 10,82 99% 1:44,06 + 5%
                                            mine, only per cents and bar 95,77 10,92 99% 1:46,94 + 8%
                                            mine, %, bar and lines 125,59 14,80 99% 2:20,83 +43%
                                            2html from vim-7.2.303 97,34 1,16 99% 1:38,64 + 0%
                                            your 2html, no progress 77,31 0,99 99% 1:18,55 -20%
                                            your 2html, with progress 100,57 1,20 99% 1:42,76 + 4%

                                            Commands:
                                            # mine, no progress
                                            time vim messages -c 'set ft=messages | execute "%FormatCommand format html" | qa!'
                                            # mine, only % and bar
                                            time vim messages -c 'set ft=messages | let g:formatOptions={"ShowProgress": 1} | execute "%FormatCommand format
                                            html" | qa!'
                                            # mine, %, bar and lines
                                            time vim messages -c 'set ft=messages | let g:formatOptions={"ShowProgress": 2} | execute "%FormatCommand format
                                            html" | qa!'
                                            # 2html, your with progress and 2html from vim-7.2.303
                                            # Difference is that in second case I created a symlink to your file in
                                            # .vim/syntax directory
                                            time vim messages -c 'set ft=messages | execute "TOhtml" | qa!'
                                            # your 2html, with progress
                                            time vim messages -c 'set ft=messages | let g:html_no_progress=1 | execute "TOhtml" | qa!'

                                            You can download messages file here: (1,1M uncompressed, 77K compressed):
                                            http://kp-pav.narod.ru/messages.xz
                                            My script is located here (see version 1.2):
                                            http://www.vim.org/scripts/script.php?script_id=3113

                                            I do not know what exactly is a problem, but your progress is too slow.


                                            Текст сообщения:
                                            > On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <fritzophrenic@...> wrote:
                                            > > This version is still not fast enough though. It is about 30% slower
                                            > > when the progress bar is enabled than when it is disabled. While I
                                            > > consider it a good tradeoff in most cases, we could certainly make it
                                            > > better.
                                            > >
                                            > > It would probably be faster to pre-calculate the line numbers needed
                                            > > to advance the progress bar rather than doing a bunch of
                                            > > floating-point math every cycle.
                                            >
                                            > I've attached a new version which pre-calculates the (integer) line
                                            > numbers needed to advance the progress bar. Now all the floating point
                                            > math is done once, up front.
                                            >
                                            > The difference is not really very perceptible. I timed the execution
                                            > on two files. First, I did the 5148-line autoload/phpcomplete.vim
                                            > script. Timings were as follows on my laptop:
                                            >
                                            > progress disabled:
                                            > average: 46 seconds
                                            >
                                            > floating-point progress:
                                            > average: 61 seconds
                                            > slowdown: 15 seconds longer than without progress bar
                                            > percentage: 33% longer than without progress bar
                                            >
                                            > precalculated progress:
                                            > average: 62 seconds
                                            > slowdown: 16 seconds longer than without progress bar
                                            > percentage: 35% longer than without progress bar
                                            >
                                            > Next I did a 33258-line C code file:
                                            >
                                            > progress disabled:
                                            > average: 691 seconds
                                            >
                                            > floating-point progress:
                                            > average: 716 seconds
                                            > slowdown: 25 seconds longer than without progress bar
                                            > percentage: 4% longer than without progress bar
                                            >
                                            > precalculated progress:
                                            > average: 711 seconds
                                            > slowdown: 20 seconds longer than without progress bar
                                            > percentage: 3% longer than without progress bar
                                            >
                                            > I also did a number of very small sections of files (my usual use case
                                            > for 2html) and did not notice any significant slowing; it only takes
                                            > 1-2 seconds longer for a 100 or 200 line selection.
                                            >
                                            > I take a few things from this.
                                            >
                                            > First of all, I don't think we'll get much performance improvement
                                            > with this method. I do not know whether it is setting the status line
                                            > and redrawing it, or whether it is the use of the object-oriented
                                            > style functions, but it would probably require a different approach to
                                            > get a significant speedup. I certainly like the look a lot better than
                                            > the echo method, even if we could get echon working. Is a 10-20 second
                                            > slow-down acceptable on large numbers of lines, if the normal
                                            > execution time is measured in minutes anyway? To me, it certainly is.
                                            > If something is going to be taking more than a few minutes, I want a
                                            > progress bar to tell me whether it's worth letting it continue. Since
                                            > the slow-down can be significant for midsize files, we will certainly
                                            > need to mention in the :help that disabling the progress bar will make
                                            > the conversion faster. Maybe we should only show the progress bar
                                            > after some amount of time has elapsed? We could suppress the
                                            > redrawstatus until 10 seconds have passed, or something like that. Any
                                            > thoughts?
                                            >
                                            > Secondly, the precalculated version is not really any faster than the
                                            > full floating-point calculation every cycle. I don't really have an
                                            > opinion of which method gives more readable code. Does anyone else
                                            > have any opinions on which version to keep? I think it would be
                                            > possible to do away with floating point calculations entirely using
                                            > the precalculated version; currently floating point is only used in
                                            > the calculate_ticks function. This might be desireable so that we can
                                            > remove the dependence on the +float feature, which is not marked with
                                            > a "smallest version" indicator in :help +feature-list. This apparently
                                            > means it is "system dependent". Does this mean float is pretty much
                                            > always included, unless it is explicitly removed? How common are Vims
                                            > without floating-point support? I already added use of the split()
                                            > function, which was added in version 7, so this won't work on really
                                            > old Vims...but do we want to support Vim 7.1 and earlier?
                                            >
                                          • ZyX
                                            Ответ на сообщение «Re: Progress indicator for :TOhtml command», присланное в 19:39:15 05 июня 2010, Суббота,
                                            Message 21 of 28 , Jun 5, 2010
                                            • 0 Attachment
                                              Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
                                              присланное в 19:39:15 05 июня 2010, Суббота,
                                              отправитель Benjamin Fritz:

                                              It occures that the problem is not floating-point math: the attached patch
                                              removes this math but does not add any perfomance. It also removes recalculating
                                              progress bar width (you just used used some generic progress bar?) and
                                              needs_redraw. Also, why you forbid profiling progress bar functions? It is also
                                              fixed.

                                              Текст сообщения:
                                              > On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <fritzophrenic@...>
                                              wrote:
                                              > > This version is still not fast enough though. It is about 30% slower
                                              > > when the progress bar is enabled than when it is disabled. While I
                                              > > consider it a good tradeoff in most cases, we could certainly make it
                                              > > better.
                                              > >
                                              > > It would probably be faster to pre-calculate the line numbers needed
                                              > > to advance the progress bar rather than doing a bunch of
                                              > > floating-point math every cycle.
                                              >
                                              > I've attached a new version which pre-calculates the (integer) line
                                              > numbers needed to advance the progress bar. Now all the floating point
                                              > math is done once, up front.
                                              >
                                              > The difference is not really very perceptible. I timed the execution
                                              > on two files. First, I did the 5148-line autoload/phpcomplete.vim
                                              > script. Timings were as follows on my laptop:
                                              >
                                              > progress disabled:
                                              > average: 46 seconds
                                              >
                                              > floating-point progress:
                                              > average: 61 seconds
                                              > slowdown: 15 seconds longer than without progress bar
                                              > percentage: 33% longer than without progress bar
                                              >
                                              > precalculated progress:
                                              > average: 62 seconds
                                              > slowdown: 16 seconds longer than without progress bar
                                              > percentage: 35% longer than without progress bar
                                              >
                                              > Next I did a 33258-line C code file:
                                              >
                                              > progress disabled:
                                              > average: 691 seconds
                                              >
                                              > floating-point progress:
                                              > average: 716 seconds
                                              > slowdown: 25 seconds longer than without progress bar
                                              > percentage: 4% longer than without progress bar
                                              >
                                              > precalculated progress:
                                              > average: 711 seconds
                                              > slowdown: 20 seconds longer than without progress bar
                                              > percentage: 3% longer than without progress bar
                                              >
                                              > I also did a number of very small sections of files (my usual use case
                                              > for 2html) and did not notice any significant slowing; it only takes
                                              > 1-2 seconds longer for a 100 or 200 line selection.
                                              >
                                              > I take a few things from this.
                                              >
                                              > First of all, I don't think we'll get much performance improvement
                                              > with this method. I do not know whether it is setting the status line
                                              > and redrawing it, or whether it is the use of the object-oriented
                                              > style functions, but it would probably require a different approach to
                                              > get a significant speedup. I certainly like the look a lot better than
                                              > the echo method, even if we could get echon working. Is a 10-20 second
                                              > slow-down acceptable on large numbers of lines, if the normal
                                              > execution time is measured in minutes anyway? To me, it certainly is.
                                              > If something is going to be taking more than a few minutes, I want a
                                              > progress bar to tell me whether it's worth letting it continue. Since
                                              > the slow-down can be significant for midsize files, we will certainly
                                              > need to mention in the :help that disabling the progress bar will make
                                              > the conversion faster. Maybe we should only show the progress bar
                                              > after some amount of time has elapsed? We could suppress the
                                              > redrawstatus until 10 seconds have passed, or something like that. Any
                                              > thoughts?
                                              >
                                              > Secondly, the precalculated version is not really any faster than the
                                              > full floating-point calculation every cycle. I don't really have an
                                              > opinion of which method gives more readable code. Does anyone else
                                              > have any opinions on which version to keep? I think it would be
                                              > possible to do away with floating point calculations entirely using
                                              > the precalculated version; currently floating point is only used in
                                              > the calculate_ticks function. This might be desireable so that we can
                                              > remove the dependence on the +float feature, which is not marked with
                                              > a "smallest version" indicator in :help +feature-list. This apparently
                                              > means it is "system dependent". Does this mean float is pretty much
                                              > always included, unless it is explicitly removed? How common are Vims
                                              > without floating-point support? I already added use of the split()
                                              > function, which was added in version 7, so this won't work on really
                                              > old Vims...but do we want to support Vim 7.1 and earlier?
                                              >
                                            • Ben Fritz
                                              ... I m sorry I don t follow. You re saying that a 4% increase in time for the progress bar, and a 20% decrease without the progress bar, is too slow ? And
                                              Message 22 of 28 , Jun 5, 2010
                                              • 0 Attachment
                                                On Jun 5, 6:26 pm, ZyX <zyx....@...> wrote:
                                                > A small benchmark for your and mine scripts:
                                                >                               user    system  cpu  total      Relative
                                                > mine, no progress              93,07  10,82   99%  1:44,06    + 5%
                                                > mine, only per cents and bar   95,77  10,92   99%  1:46,94    + 8%
                                                > mine, %, bar and lines        125,59  14,80   99%  2:20,83    +43%
                                                > 2html from vim-7.2.303         97,34   1,16   99%  1:38,64    + 0%
                                                > your 2html, no progress        77,31   0,99   99%  1:18,55    -20%
                                                > your 2html, with progress     100,57   1,20   99%  1:42,76    + 4%
                                                >
                                                > [Snip]
                                                >
                                                > I do not know what exactly is a problem, but your progress is too slow.
                                                >

                                                I'm sorry I don't follow. You're saying that a 4% increase in time for
                                                the progress bar, and a 20% decrease without the progress bar, is "too
                                                slow"?

                                                And you're proposing changes that make it an 8% increase with the
                                                progress bar, or a 5% increase without?

                                                --
                                                You received this message from the "vim_dev" maillist.
                                                Do not top-post! Type your reply below the text you are replying to.
                                                For more information, visit http://www.vim.org/maillist.php
                                              • Benjamin Fritz
                                                ... Yes, I did not expect any performance gains from removing the little bit of remaining floating point, since it is just up to 100 calculations done once at
                                                Message 23 of 28 , Jun 5, 2010
                                                • 0 Attachment
                                                  On Jun 5, 8:10 pm, ZyX <zyx....@...> wrote:
                                                  >
                                                  > It occures that the problem is not floating-point math: the attached patch
                                                  > removes this math but does not add any perfomance.
                                                  >

                                                  Yes, I did not expect any performance gains from removing the little
                                                  bit of remaining floating point, since it is just up to 100
                                                  calculations done once at the start and thereafter only when the
                                                  window changes size. It is a good idea to remove, because as you point
                                                  out, that amount of precision is probably unnecessary, and it would
                                                  just introduce another dependency.

                                                  > It also removes recalculating
                                                  > progress bar width (you just used used some generic progress bar?) and
                                                  > needs_redraw.

                                                  Yes, we did use a generic progress bar as the starting point for this.
                                                  However, I think it IS necessary to recalculate the progress bar
                                                  width. This is done so that if the user changes window sizes, the
                                                  progress bar will be updated accordingly. We don't want a progress bar
                                                  that is too big to fit in the window, or smaller than needed for
                                                  decent viewing. With your patch, if you start with the gvim window
                                                  maximized, then restore the window to a smaller size, Vim goes blank
                                                  until the next progress bar update, and then the progress bar is too
                                                  large to fit on the screen and is truncated. This is not desirable,
                                                  but perhaps it would acceptable if the performance gains are great
                                                  enough. This does not seem to be the case, because I added back in the
                                                  size recalculation with no noticeable performance hit.

                                                  The needs_redraw was done in order to allow us to call redrawstatus on
                                                  the correct window. :help redrawstatus says that it redraws the status
                                                  line for the *current window* only unless you use redrawstatus! which
                                                  redraws all windows. In practice, however, it does not seem to matter
                                                  which window we use it in. Why is this?

                                                  > Also, why you forbid profiling progress bar functions? It is also
                                                  > fixed.
                                                  >

                                                  Good catch, that's certainly something to include going forward.

                                                  There is a slight speed gain from your patch, however there is a
                                                  mistaken assumption in the way you update the progress bar. Your code
                                                  assumes that the progress bar will only ever update by one tick at a
                                                  time. Updating the progress bar without your patch calculates the
                                                  entire string every time, using repeat(). Your update simply adds one
                                                  to the colored string of spaces, and subtracts one from the uncolored.
                                                  This does not work if the user folds away some text and does not use
                                                  dynamic folding, it does not work when there are fewer than 100 lines
                                                  in the text to convert, and it does not work for the second use of the
                                                  progress bar, where there are usually fewer that 100 highlight groups
                                                  to process.

                                                  I corrected this problem and initially, the performance still seemed
                                                  to be improved over the previous version. However, I noticed afterward
                                                  that part of the patch removes the "sleep 100m" from the "processing
                                                  classes" step. I took this line out of the original script for a fair
                                                  comparison, and got the following timings, converting
                                                  autoload/netrw.vim (7764 lines) with dynamic folding enabled:

                                                  Before patch: 50 seconds
                                                  Patch from ZyX: 49 seconds
                                                  Fixed patch: 51 seconds

                                                  So, it looks like the patch is actually no faster, and potentially
                                                  slightly slower than the precalculated version.

                                                  I have therefore attached an updated version of my last submission,
                                                  which removes floating point from the calculate_ticks function, and
                                                  incorporates some of the other improvements from ZyX.

                                                  This version takes 50 seconds to convert netrw, if I comment out the
                                                  sleep 100 line. Do we want this line in the code? Without it, if there
                                                  are not very many highlight groups to process, the "processing
                                                  classes" bar flashes by without being seen. This happens anyway for
                                                  very small selections. I don't know how I feel about deliberately
                                                  slowing down the execution. I have left it commented out for now.

                                                  I am very curious about this:

                                                  " Note that you must use len(split) instead of len() if you want to use
                                                  " unicode in title
                                                  let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2

                                                  Can someone explain the problem described in the comment a little
                                                  better? And why does the split on '\zs' work to fix the problem?

                                                  --
                                                  You received this message from the "vim_dev" maillist.
                                                  Do not top-post! Type your reply below the text you are replying to.
                                                  For more information, visit http://www.vim.org/maillist.php
                                                • ZyX
                                                  Ответ на сообщение «Re: Progress indicator for :TOhtml command», присланное в 10:59:42 06 июня 2010, Воскресенье,
                                                  Message 24 of 28 , Jun 6, 2010
                                                  • 0 Attachment
                                                    Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
                                                    присланное в 10:59:42 06 июня 2010, Воскресенье,
                                                    отправитель Benjamin Fritz:

                                                    The reason why I say that progress bar is too slow is that my script does not
                                                    suffer from performance decrease unless you make it redraw on each line. I will
                                                    add size recalculation for my script too (I removed it from your script because
                                                    I did not realize that while user can do nothing in vim he still can resize the
                                                    terminal), but I do not think that this will add any performance penalty.

                                                    > I am very curious about this:
                                                    >
                                                    > " Note that you must use len(split) instead of len() if you want to use
                                                    > " unicode in title
                                                    > let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
                                                    >
                                                    > Can someone explain the problem described in the comment a little
                                                    > better? And why does the split on '\zs' work to fix the problem?
                                                    That is because len(str) measures byte length of C string, while len(split) first
                                                    splits the string into a list of characters and then measures the length of
                                                    character list. If there are non-latin1 Unicode symbols and encoding is a
                                                    multibyte one then length of character list is not equal to bytes count of C
                                                    string.

                                                    Текст сообщения:
                                                    > On Jun 5, 8:10 pm, ZyX <zyx....@...> wrote:
                                                    > > It occures that the problem is not floating-point math: the attached
                                                    > > patch removes this math but does not add any perfomance.
                                                    >
                                                    > Yes, I did not expect any performance gains from removing the little
                                                    > bit of remaining floating point, since it is just up to 100
                                                    > calculations done once at the start and thereafter only when the
                                                    > window changes size. It is a good idea to remove, because as you point
                                                    > out, that amount of precision is probably unnecessary, and it would
                                                    > just introduce another dependency.
                                                    >
                                                    > > It also removes recalculating
                                                    > > progress bar width (you just used used some generic progress bar?) and
                                                    > > needs_redraw.
                                                    >
                                                    > Yes, we did use a generic progress bar as the starting point for this.
                                                    > However, I think it IS necessary to recalculate the progress bar
                                                    > width. This is done so that if the user changes window sizes, the
                                                    > progress bar will be updated accordingly. We don't want a progress bar
                                                    > that is too big to fit in the window, or smaller than needed for
                                                    > decent viewing. With your patch, if you start with the gvim window
                                                    > maximized, then restore the window to a smaller size, Vim goes blank
                                                    > until the next progress bar update, and then the progress bar is too
                                                    > large to fit on the screen and is truncated. This is not desirable,
                                                    > but perhaps it would acceptable if the performance gains are great
                                                    > enough. This does not seem to be the case, because I added back in the
                                                    > size recalculation with no noticeable performance hit.
                                                    >
                                                    > The needs_redraw was done in order to allow us to call redrawstatus on
                                                    > the correct window. :help redrawstatus says that it redraws the status
                                                    > line for the *current window* only unless you use redrawstatus! which
                                                    > redraws all windows. In practice, however, it does not seem to matter
                                                    > which window we use it in. Why is this?
                                                    >
                                                    > > Also, why you forbid profiling progress bar functions? It is also
                                                    > > fixed.
                                                    >
                                                    > Good catch, that's certainly something to include going forward.
                                                    >
                                                    > There is a slight speed gain from your patch, however there is a
                                                    > mistaken assumption in the way you update the progress bar. Your code
                                                    > assumes that the progress bar will only ever update by one tick at a
                                                    > time. Updating the progress bar without your patch calculates the
                                                    > entire string every time, using repeat(). Your update simply adds one
                                                    > to the colored string of spaces, and subtracts one from the uncolored.
                                                    > This does not work if the user folds away some text and does not use
                                                    > dynamic folding, it does not work when there are fewer than 100 lines
                                                    > in the text to convert, and it does not work for the second use of the
                                                    > progress bar, where there are usually fewer that 100 highlight groups
                                                    > to process.
                                                    >
                                                    > I corrected this problem and initially, the performance still seemed
                                                    > to be improved over the previous version. However, I noticed afterward
                                                    > that part of the patch removes the "sleep 100m" from the "processing
                                                    > classes" step. I took this line out of the original script for a fair
                                                    > comparison, and got the following timings, converting
                                                    > autoload/netrw.vim (7764 lines) with dynamic folding enabled:
                                                    >
                                                    > Before patch: 50 seconds
                                                    > Patch from ZyX: 49 seconds
                                                    > Fixed patch: 51 seconds
                                                    >
                                                    > So, it looks like the patch is actually no faster, and potentially
                                                    > slightly slower than the precalculated version.
                                                    >
                                                    > I have therefore attached an updated version of my last submission,
                                                    > which removes floating point from the calculate_ticks function, and
                                                    > incorporates some of the other improvements from ZyX.
                                                    >
                                                    > This version takes 50 seconds to convert netrw, if I comment out the
                                                    > sleep 100 line. Do we want this line in the code? Without it, if there
                                                    > are not very many highlight groups to process, the "processing
                                                    > classes" bar flashes by without being seen. This happens anyway for
                                                    > very small selections. I don't know how I feel about deliberately
                                                    > slowing down the execution. I have left it commented out for now.
                                                    >
                                                    > I am very curious about this:
                                                    >
                                                    > " Note that you must use len(split) instead of len() if you want to use
                                                    > " unicode in title
                                                    > let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
                                                    >
                                                    > Can someone explain the problem described in the comment a little
                                                    > better? And why does the split on '\zs' work to fix the problem?
                                                    >
                                                  • ZyX
                                                    Ответ на сообщение «Re: Progress indicator for :TOhtml command», присланное в 10:59:42 06 июня 2010, Воскресенье,
                                                    Message 25 of 28 , Jun 6, 2010
                                                    • 0 Attachment
                                                      Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
                                                      присланное в 10:59:42 06 июня 2010, Воскресенье,
                                                      отправитель Benjamin Fritz:

                                                      It is odd: the only problem in your script is redrawstatus which is called only
                                                      100 times (without styles, 109 with) (>21 seconds), while in my script
                                                      redrawstatus called 328 times takes less than a second.

                                                      Second problem with the whole 2html is buffer switching, I think you should
                                                      consider instead of doing constant switches, save every line in a List and only
                                                      after everything is finished create a new buffer and call setline(1, s:list).
                                                      Note that new versions of my script are faster (but not much) then your 2html
                                                      because I use this technique.

                                                      And, why do you calculate length of the title at each progressbarupdate?
                                                      Attached patch fixes this and the case when there is no space for progress bar.

                                                      Текст сообщения:
                                                      > On Jun 5, 8:10 pm, ZyX <zyx....@...> wrote:
                                                      > > It occures that the problem is not floating-point math: the attached
                                                      > > patch removes this math but does not add any perfomance.
                                                      >
                                                      > Yes, I did not expect any performance gains from removing the little
                                                      > bit of remaining floating point, since it is just up to 100
                                                      > calculations done once at the start and thereafter only when the
                                                      > window changes size. It is a good idea to remove, because as you point
                                                      > out, that amount of precision is probably unnecessary, and it would
                                                      > just introduce another dependency.
                                                      >
                                                      > > It also removes recalculating
                                                      > > progress bar width (you just used used some generic progress bar?) and
                                                      > > needs_redraw.
                                                      >
                                                      > Yes, we did use a generic progress bar as the starting point for this.
                                                      > However, I think it IS necessary to recalculate the progress bar
                                                      > width. This is done so that if the user changes window sizes, the
                                                      > progress bar will be updated accordingly. We don't want a progress bar
                                                      > that is too big to fit in the window, or smaller than needed for
                                                      > decent viewing. With your patch, if you start with the gvim window
                                                      > maximized, then restore the window to a smaller size, Vim goes blank
                                                      > until the next progress bar update, and then the progress bar is too
                                                      > large to fit on the screen and is truncated. This is not desirable,
                                                      > but perhaps it would acceptable if the performance gains are great
                                                      > enough. This does not seem to be the case, because I added back in the
                                                      > size recalculation with no noticeable performance hit.
                                                      >
                                                      > The needs_redraw was done in order to allow us to call redrawstatus on
                                                      > the correct window. :help redrawstatus says that it redraws the status
                                                      > line for the *current window* only unless you use redrawstatus! which
                                                      > redraws all windows. In practice, however, it does not seem to matter
                                                      > which window we use it in. Why is this?
                                                      >
                                                      > > Also, why you forbid profiling progress bar functions? It is also
                                                      > > fixed.
                                                      >
                                                      > Good catch, that's certainly something to include going forward.
                                                      >
                                                      > There is a slight speed gain from your patch, however there is a
                                                      > mistaken assumption in the way you update the progress bar. Your code
                                                      > assumes that the progress bar will only ever update by one tick at a
                                                      > time. Updating the progress bar without your patch calculates the
                                                      > entire string every time, using repeat(). Your update simply adds one
                                                      > to the colored string of spaces, and subtracts one from the uncolored.
                                                      > This does not work if the user folds away some text and does not use
                                                      > dynamic folding, it does not work when there are fewer than 100 lines
                                                      > in the text to convert, and it does not work for the second use of the
                                                      > progress bar, where there are usually fewer that 100 highlight groups
                                                      > to process.
                                                      >
                                                      > I corrected this problem and initially, the performance still seemed
                                                      > to be improved over the previous version. However, I noticed afterward
                                                      > that part of the patch removes the "sleep 100m" from the "processing
                                                      > classes" step. I took this line out of the original script for a fair
                                                      > comparison, and got the following timings, converting
                                                      > autoload/netrw.vim (7764 lines) with dynamic folding enabled:
                                                      >
                                                      > Before patch: 50 seconds
                                                      > Patch from ZyX: 49 seconds
                                                      > Fixed patch: 51 seconds
                                                      >
                                                      > So, it looks like the patch is actually no faster, and potentially
                                                      > slightly slower than the precalculated version.
                                                      >
                                                      > I have therefore attached an updated version of my last submission,
                                                      > which removes floating point from the calculate_ticks function, and
                                                      > incorporates some of the other improvements from ZyX.
                                                      >
                                                      > This version takes 50 seconds to convert netrw, if I comment out the
                                                      > sleep 100 line. Do we want this line in the code? Without it, if there
                                                      > are not very many highlight groups to process, the "processing
                                                      > classes" bar flashes by without being seen. This happens anyway for
                                                      > very small selections. I don't know how I feel about deliberately
                                                      > slowing down the execution. I have left it commented out for now.
                                                      >
                                                      > I am very curious about this:
                                                      >
                                                      > " Note that you must use len(split) instead of len() if you want to use
                                                      > " unicode in title
                                                      > let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
                                                      >
                                                      > Can someone explain the problem described in the comment a little
                                                      > better? And why does the split on '\zs' work to fix the problem?
                                                      >
                                                    • ZyX
                                                      Ответ на сообщение «Re: Progress indicator for :TOhtml command», присланное в 13:03:23 06 июня 2010, Воскресенье,
                                                      Message 26 of 28 , Jun 6, 2010
                                                      • 0 Attachment
                                                        Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
                                                        присланное в 13:03:23 06 июня 2010, Воскресенье,
                                                        отправитель ZyX:

                                                        Yes, buffer switching is the problem: attached patch uses my technique (save
                                                        everything in a list, not in a buffer) and here are the results:

                                                        My script:
                                                        1:05,09 w/o progress
                                                        1:08,40 ShowProgress=1
                                                        1:20,59 ShowProgress=2
                                                        Your 2html:
                                                        1:19,67 w/o progress
                                                        1:44,74 with progress
                                                        Patched 2html:
                                                        1:03,51 w/o progress
                                                        1:05,08 with progress

                                                        Apply patch to your 2html, not to previously patched version.

                                                        Текст сообщения:
                                                        > Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
                                                        > присланное в 10:59:42 06 июня 2010, Воскресенье,
                                                        > отправитель Benjamin Fritz:
                                                        >
                                                        > It is odd: the only problem in your script is redrawstatus which is called
                                                        > only 100 times (without styles, 109 with) (>21 seconds), while in my
                                                        > script redrawstatus called 328 times takes less than a second.
                                                        >
                                                        > Second problem with the whole 2html is buffer switching, I think you should
                                                        > consider instead of doing constant switches, save every line in a List and
                                                        > only after everything is finished create a new buffer and call setline(1,
                                                        > s:list). Note that new versions of my script are faster (but not much)
                                                        > then your 2html because I use this technique.
                                                        >
                                                        > And, why do you calculate length of the title at each progressbarupdate?
                                                        > Attached patch fixes this and the case when there is no space for progress
                                                        > bar.
                                                        >
                                                        > Текст сообщения:
                                                        > > On Jun 5, 8:10 pm, ZyX <zyx....@...> wrote:
                                                        > > > It occures that the problem is not floating-point math: the attached
                                                        > > > patch removes this math but does not add any perfomance.
                                                        > >
                                                        > > Yes, I did not expect any performance gains from removing the little
                                                        > > bit of remaining floating point, since it is just up to 100
                                                        > > calculations done once at the start and thereafter only when the
                                                        > > window changes size. It is a good idea to remove, because as you point
                                                        > > out, that amount of precision is probably unnecessary, and it would
                                                        > > just introduce another dependency.
                                                        > >
                                                        > > > It also removes recalculating
                                                        > > > progress bar width (you just used used some generic progress bar?) and
                                                        > > > needs_redraw.
                                                        > >
                                                        > > Yes, we did use a generic progress bar as the starting point for this.
                                                        > > However, I think it IS necessary to recalculate the progress bar
                                                        > > width. This is done so that if the user changes window sizes, the
                                                        > > progress bar will be updated accordingly. We don't want a progress bar
                                                        > > that is too big to fit in the window, or smaller than needed for
                                                        > > decent viewing. With your patch, if you start with the gvim window
                                                        > > maximized, then restore the window to a smaller size, Vim goes blank
                                                        > > until the next progress bar update, and then the progress bar is too
                                                        > > large to fit on the screen and is truncated. This is not desirable,
                                                        > > but perhaps it would acceptable if the performance gains are great
                                                        > > enough. This does not seem to be the case, because I added back in the
                                                        > > size recalculation with no noticeable performance hit.
                                                        > >
                                                        > > The needs_redraw was done in order to allow us to call redrawstatus on
                                                        > > the correct window. :help redrawstatus says that it redraws the status
                                                        > > line for the *current window* only unless you use redrawstatus! which
                                                        > > redraws all windows. In practice, however, it does not seem to matter
                                                        > > which window we use it in. Why is this?
                                                        > >
                                                        > > > Also, why you forbid profiling progress bar functions? It is also
                                                        > > > fixed.
                                                        > >
                                                        > > Good catch, that's certainly something to include going forward.
                                                        > >
                                                        > > There is a slight speed gain from your patch, however there is a
                                                        > > mistaken assumption in the way you update the progress bar. Your code
                                                        > > assumes that the progress bar will only ever update by one tick at a
                                                        > > time. Updating the progress bar without your patch calculates the
                                                        > > entire string every time, using repeat(). Your update simply adds one
                                                        > > to the colored string of spaces, and subtracts one from the uncolored.
                                                        > > This does not work if the user folds away some text and does not use
                                                        > > dynamic folding, it does not work when there are fewer than 100 lines
                                                        > > in the text to convert, and it does not work for the second use of the
                                                        > > progress bar, where there are usually fewer that 100 highlight groups
                                                        > > to process.
                                                        > >
                                                        > > I corrected this problem and initially, the performance still seemed
                                                        > > to be improved over the previous version. However, I noticed afterward
                                                        > > that part of the patch removes the "sleep 100m" from the "processing
                                                        > > classes" step. I took this line out of the original script for a fair
                                                        > > comparison, and got the following timings, converting
                                                        > > autoload/netrw.vim (7764 lines) with dynamic folding enabled:
                                                        > >
                                                        > > Before patch: 50 seconds
                                                        > > Patch from ZyX: 49 seconds
                                                        > > Fixed patch: 51 seconds
                                                        > >
                                                        > > So, it looks like the patch is actually no faster, and potentially
                                                        > > slightly slower than the precalculated version.
                                                        > >
                                                        > > I have therefore attached an updated version of my last submission,
                                                        > > which removes floating point from the calculate_ticks function, and
                                                        > > incorporates some of the other improvements from ZyX.
                                                        > >
                                                        > > This version takes 50 seconds to convert netrw, if I comment out the
                                                        > > sleep 100 line. Do we want this line in the code? Without it, if there
                                                        > > are not very many highlight groups to process, the "processing
                                                        > > classes" bar flashes by without being seen. This happens anyway for
                                                        > > very small selections. I don't know how I feel about deliberately
                                                        > > slowing down the execution. I have left it commented out for now.
                                                        > >
                                                        > > I am very curious about this:
                                                        > >
                                                        > > " Note that you must use len(split) instead of len() if you want to use
                                                        > > " unicode in title
                                                        > > let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
                                                        > >
                                                        > > Can someone explain the problem described in the comment a little
                                                        > > better? And why does the split on '\zs' work to fix the problem?
                                                        >
                                                      • Benjamin Fritz
                                                        ... Very nice. This is a huge performance boost, and the times are similar with and without the progress bar even with my big 33000 line C file which I used
                                                        Message 27 of 28 , Jun 7, 2010
                                                        • 0 Attachment
                                                          On Sun, Jun 6, 2010 at 5:10 AM, ZyX <zyx.vim@...> wrote:
                                                          >
                                                          > Yes, buffer switching is the problem: attached patch uses my technique (save
                                                          > everything in a list, not in a buffer) and here are the results:
                                                          >
                                                          > My script:
                                                          > 1:05,09 w/o progress
                                                          > 1:08,40 ShowProgress=1
                                                          > 1:20,59 ShowProgress=2
                                                          > Your 2html:
                                                          > 1:19,67 w/o progress
                                                          > 1:44,74 with progress
                                                          > Patched 2html:
                                                          > 1:03,51 w/o progress
                                                          > 1:05,08 with progress
                                                          >

                                                          Very nice. This is a huge performance boost, and the times are similar
                                                          with and without the progress bar even with my big 33000 line C file
                                                          which I used previously.

                                                          I think it's about ready now. I've added another progress bar for the
                                                          time taken to collect fold information for dynamic folding, and
                                                          corrected a few minor bugs in the patch related to dynamic folding. I
                                                          did end up adding back in a :sleep to the class processing loop, but I
                                                          reduced the time it sleeps. I'm certainly open to removing this.

                                                          I've attached the whole file so we don't get into a "which patches do
                                                          I need?" quagmire.

                                                          --
                                                          You received this message from the "vim_dev" maillist.
                                                          Do not top-post! Type your reply below the text you are replying to.
                                                          For more information, visit http://www.vim.org/maillist.php
                                                        Your message has been successfully submitted and would be delivered to recipients shortly.