Loading ...
Sorry, an error occurred while loading the content.

Bug Sucks CPU Dry for Long XML lines

Expand Messages
  • RParker@weather.com
    Using the default XML syntax highlighting, I get extremely slow editing on .xml files that contain very long lines. This worked fine on vim 5. I have patched
    Message 1 of 8 , Nov 14, 2001
    • 0 Attachment
      Using the default XML syntax highlighting, I get extremely slow editing on
      ".xml" files that contain very long lines.

      This worked fine on vim 5.

      I have patched to the latest vim 6.0.93

      On a dual P3-1GHz with 1G running Linux 2.2.19, a 10,000 char line takes a
      couple of seconds to load and search. A file with a single 250,000 char line
      take 9 seconds to load, and I gave up a simple XML tag search after 100% cpu (on
      one of my 2 chips) for 3 minutes.

      Did I hear somebody say that a quarter meg without a newline makes XML tough to
      read? I'm with you, but that's what I'm stuck with...

      If I remove the ".xml" file extension, normal performance returns. Likewise, if
      I use a vim-5 binary, there is no problem even with a big ".xml" file.

      - Randy
    • Ron Aaron
      writes: ... Hi, Randy - Try a TeX file on a slow machine -- same problem. The fact is, the syntax highlighting is quite slow, noticeably
      Message 2 of 8 , Nov 14, 2001
      • 0 Attachment
        <RParker@...> writes:
        ...

        >If I remove the ".xml" file extension, normal performance returns. Likewise, if
        >I use a vim-5 binary, there is no problem even with a big ".xml" file.

        Hi, Randy -

        Try a TeX file on a slow machine -- same problem.

        The fact is, the syntax highlighting is quite slow, noticeably so on my Linux
        200 Mhz laptop. Any suggestions as to how to speed up the regexp engine
        (which the syntax matcher uses extensively) would be welcome. AS would some
        sort of profile run showing where vim is spending all that time.

        Can you run an instrumented build?

        Best regards,
        Ron
      • Dan Sharp
        ... Turn off syntax highlighting (:syn off) and see if the situation improves. If it does, then one of the highlighting definitions in the xml.vim syntax file
        Message 3 of 8 , Nov 14, 2001
        • 0 Attachment
          >From: RParker@...
          >
          >Using the default XML syntax highlighting, I get extremely slow editing on
          >".xml" files that contain very long lines.

          Turn off syntax highlighting (:syn off) and see if the situation improves.
          If it does, then one of the highlighting definitions in the xml.vim syntax
          file is causing the problem. You can download the latest version (dated
          November 3) from http://www.zellner.org/vim/syntax/xml.vim and see if things
          have improved. If not, if you have some time maybe you can try removing
          lines from the syntax file to find which line is causing the problem, then
          let Johannes Zellner know, to see if the regexp can be improved.

          Dan Sharp

          _________________________________________________________________
          Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
        • RParker@weather.com
          ... and ... Yes, but my point is that it worked 1,000 times faster in vim-5. ... As I said, without syntax highlighting, the performance is normal again. (ie,
          Message 4 of 8 , Nov 14, 2001
          • 0 Attachment
            A couple of helpful people suggested:

            >> syntax highlighting [is just] slow [even for other apps like TeX]...

            and

            >> try to identify the offending [xml.vim] line...

            Yes, but my point is that it worked 1,000 times faster in vim-5.

            >> Turn off syntax highlighting and see if the situation improves...

            As I said, without syntax highlighting, the performance is normal again. (ie,
            1,000 times faster than with syntax highlighting).

            It seems to me the question is "what changed from vim-5?"

            I'll try Johannes Zellner's latest "xml.vim" anyway. I want to make sure I
            don't end up running both the default syntax file, and Johannes' latest one, one
            after the other. I want to replace the default one entirely.

            I notice Johannes' file contains the NOTE:
            "for large files, folding can be pretty slow, especially for large files...."

            I find that even simple searches, occurring long after the file is loaded, take
            impossible amounts of time. Say, 3 minutes of 100% CPU on a 1,000 MHz P3 with
            1,000 MB of memory to find 4-char tags (with search target highlighting turned
            on).

            - Randy
          • Charles E. Campbell
            ... Sigh, tis true enough I m afraid... ... Do you have set foldmethod=syntax somewhere? I ve found that doing that makes vim excruciatingly slow. (check out
            Message 5 of 8 , Nov 14, 2001
            • 0 Attachment
              Thus saith RParker@...:
              > >> syntax highlighting [is just] slow [even for other apps like TeX]...

              Sigh, 'tis true enough I'm afraid...

              > and
              >
              > >> try to identify the offending [xml.vim] line...
              >
              > Yes, but my point is that it worked 1,000 times faster in vim-5.
              > It seems to me the question is "what changed from vim-5?"
              >
              > I notice Johannes' file contains the NOTE:
              > "for large files, folding can be pretty slow, especially for large files...."
              ---------------------------------------------------------------------

              Do you have

              set foldmethod=syntax

              somewhere? I've found that doing that makes vim excruciatingly slow.
              (check out hideouts such as <.vimrc> <.vim/ftplugin/xml/...> etc

              Regards,
              C Campbell

              --
              Charles E Campbell, Jr, PhD _ __ __
              Goddard Space Flight Center / /_/\_\_/ /
              cec@... /_/ \/_//_/
              PGP public key: http://www.erols.com/astronaut/pgp.html
            • Benoit Cerrina
              What foldmethod do you use? It has been found that syntax folding is for some reason extremely slow. Benoit ... From: To:
              Message 6 of 8 , Nov 14, 2001
              • 0 Attachment
                What foldmethod do you use?
                It has been found that syntax folding is for some reason extremely slow.
                Benoit
                ----- Original Message -----
                From: <RParker@...>
                To: <vim-dev@...>
                Sent: Wednesday, November 14, 2001 9:10 PM
                Subject: Re: Bug Sucks CPU Dry for Long XML lines


                >
                > A couple of helpful people suggested:
                >
                > >> syntax highlighting [is just] slow [even for other apps like TeX]...
                >
                > and
                >
                > >> try to identify the offending [xml.vim] line...
                >
                > Yes, but my point is that it worked 1,000 times faster in vim-5.
                >
                > >> Turn off syntax highlighting and see if the situation improves...
                >
                > As I said, without syntax highlighting, the performance is normal again.
                (ie,
                > 1,000 times faster than with syntax highlighting).
                >
                > It seems to me the question is "what changed from vim-5?"
                >
                > I'll try Johannes Zellner's latest "xml.vim" anyway. I want to make sure
                I
                > don't end up running both the default syntax file, and Johannes' latest
                one, one
                > after the other. I want to replace the default one entirely.
                >
                > I notice Johannes' file contains the NOTE:
                > "for large files, folding can be pretty slow, especially for large
                files...."
                >
                > I find that even simple searches, occurring long after the file is loaded,
                take
                > impossible amounts of time. Say, 3 minutes of 100% CPU on a 1,000 MHz P3
                with
                > 1,000 MB of memory to find 4-char tags (with search target highlighting
                turned
                > on).
                >
                > - Randy
                >
                >
              • Dan Sharp
                ... Several things. Diff the xml.vim syntax files from 5.8 and 6.0 and you will see quite a few differences, with many additions to the 6.0 version. Vim 6.0
                Message 7 of 8 , Nov 14, 2001
                • 0 Attachment
                  >From: RParker@...
                  >
                  >It seems to me the question is "what changed from vim-5?"

                  Several things. Diff the xml.vim syntax files from 5.8 and 6.0 and you will
                  see quite a few differences, with many additions to the 6.0 version. Vim
                  6.0 introduced new regexp patterns, of which the newest xml.vim file takes
                  advantage, but perhaps not in the best way (search the vimdev archives for
                  'xml syntax' and read the messages about stack overflows / crashes in some
                  of the alpha versions due to syntax highlighting patterns). Maybe not all
                  the bugs have been worked out. Out of curiousity, can you reformat you XML
                  file into a more "readable" form and see if you still have the problems?
                  Also, does using the syntax file from 5.8 with 6.0 cause the same problems?
                  If you don't need the extra features in the 6.0 version, maybe you can get
                  by with the 5.8 version.

                  >I find that even simple searches, occurring long after the file is
                  >loaded, take impossible amounts of time. Say, 3 minutes of 100% CPU
                  >on a 1,000 MHz P3 with 1,000 MB of memory to find 4-char tags (with
                  >search target highlighting turned on).

                  That is probably because with 'incsearch' turned on the cursor is updated to
                  the nearest match with each letter you type, causing many screen redraws and
                  thus a lot of syntax parsing.

                  Dan Sharp

                  _________________________________________________________________
                  Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
                • RParker@weather.com
                  I m going to have to suspend investigation until after (..if) I deliver a release on Friday. When I generate a 25k xml file without newlines by duplicating
                  Message 8 of 8 , Nov 14, 2001
                  • 0 Attachment
                    I'm going to have to suspend investigation until after (..if) I deliver a
                    release on Friday.

                    When I generate a 25k xml file without newlines by duplicating (inserting) the
                    same 25-byte tag pair 1,000 times (this takes 30 cpu sec), the problem does not
                    occur. (insert <aaaaaaaa>1234</aaaaaaaa>ESC)

                    So I'll return to the original 250k file, with an xml tag hierarchy 5 layers
                    deep. If I do a global substitution to put a newline after each end-tag , the
                    problems disappear. (substitute xml's ">" with ">newline" (using ctrl-v to put
                    in ctrl-M)). In other words, the bug only appears on gigantic single lines, and
                    even then, only if the hierarchy is nested deeper than 1 layer.

                    There is definitely something wrong, though. Vim-5 is lightening fast for the
                    same operation. Once the file gets loaded (the original 250k file, not the 25k
                    synthetic one proposed above) (~9 sec), it takes ~8 sec to highlight a tag that
                    appears 200 times in the file. Once the highlighting completes (8 sec), just
                    hitting "n" to advance a bit over 10k in the file causes the CPU to saturate for
                    13 sec. Don't forget that this is on a 1,000MHz computer with 1,000 MB of
                    memory.

                    My guess is that the highlighting in vim-5 was applied only to the visible
                    window. In vim-6, perhaps the window-clipping was dropped. This does not
                    explain the importance of newlines... unless the clipping is now applied on a
                    line-by-line basis, instead of byte-by-byte like in vim-5. (how's that for
                    ignorant speculation?)

                    Yes, the regular expresssions have changed as well, but I suspect that is not
                    the cause of a 1,000 fold performance loss.

                    - Randy
                  Your message has been successfully submitted and would be delivered to recipients shortly.