Loading ...
Sorry, an error occurred while loading the content.

column-oriented editing

Expand Messages
  • Andrew Pimlott
    I have for a long time wanted vim to be better at editing comma-separated value and other column-oriented files. When I saw the vertical split and other
    Message 1 of 12 , Feb 3 12:46 PM
    • 0 Attachment
      I have for a long time wanted vim to be better at editing
      "comma-separated value" and other column-oriented files. When I saw
      the vertical split and other features added to vim 6, I thought I
      might be able to hack it together, but there are some pieces
      missing. So I'll describe the idea and suggest what I think is
      needed to implement it.

      (I'm sending this to vim-dev since it's largely about the
      implementation, but I can discuss the idea on vim if you think that
      would be a better place to start.)

      Of course, you can edit CSV and similar files with vim, but there
      are some drawbacks:

      - The columns aren't lined up visually
      - You can't easily tell which column you're on
      - You can't limit operations to a column
      - You have to take care of proper formatting (ie, quoting or
      escaping) by hand

      (There is a csv.vim file at vim.org, but it doesn't go very far.)

      Some people would just use a spreadsheet, but I don't find this
      satisfactory.

      - vim is much better at textual manipulation (and generally better)
      - Most spreadsheets require you to go through several steps to open
      a CSV-type file, making them tedious for quick edits
      - Most spreadsheets try to perform some unwanted interpretation of
      CSV-type files, eg auto-detecting (and thus reformatting) numeric
      columns

      Note that I'm not asking for spreadsheet features in vim. I just
      want to treat my CSV files as columns of text.

      The basic idea is to split the file into columns in some automated
      way and put each column into a vertically-split window (possibly
      with a thin horizontal header window at the top). Editing should
      mostly work as normal, except

      - columns should stay lined-up relative to each other
      - it should be easy to move between columns
      - some operations (adding or deleting a line, quitting) should act
      on all columns
      - some operations (searching, completing) should look first at the
      current column, then on the other columns
      - some data (marks, undo buffer) should be common to all columns
      - visual selection across columns... ?

      Of course, I'm willing to settle for some reasonable subset of this
      functionality.

      What I have:

      * Splitting files into columns and reassembling the columns can be
      done with a wrapper script, or inside vim. Inside vim is better,
      because it doesn't require running a special program, and vim can
      reconstruct the original file at every save. I've implemented this
      with autocommands. It's a little weird, since opening a single
      file creates many windows, non containing that file. But it seems
      to work.

      * scroll-binding keeps the windows lined-up, but it doesn't keep the
      cursor on the same row in every window. This seems like a useful
      addition. Actually, I don't see any reason not to do this by
      default when scrollbind is set, because even now the cursor is
      moved in the non-current windows when it would go off-screen.
      Keeping it on the same (relative) row in all windows would be more
      consistent.

      I think this would have to be done within vim itself. The only
      alternative would be an autocommand for cursor motions. Is that
      practical?

      * Performing operations on all columns can be done to some extent
      with mappings. One major annoyance here is that :windo command
      makes the last window current. This really should be fixed to
      make the original window current at the end. Then, you could
      simply do things like

      map o :windo o<CR>
      imap <CR> :windo <C-O>o<CR>

      But this can be worked around.

      One limitation is that I don't think you can override a builtin ex
      command. :w and :q can be handled with autocommands (though there
      is no autocommand that corresponds precisely to :q). But it would
      be nice if some other ex commands could apply to all colums.
      (Actually, I can't think of any compelling examples at the
      moment.)

      * Operations that should prefer the current column then fall back to
      other columns would probably have to be done within vim. This is
      already possible for completion, but not AFAICT for searching.
      This is probably a relatively low priority, though.

      * Data common to all columns would have to be done within vim. This
      seems to require pretty intrusive changes, but I really don't
      think I would be able to stand not being able to undo across
      columns. vim would probably have to learn the notion of several
      buffers being one "logical buffer". Of course, once you do this,
      most of the other changes above fall out naturally.

      * Visual selection across columns is just me musing.

      Any suggestions or recommendations about how I should go about this
      are welcome. In particular, ways to do things without changing vim
      itself (I'm pretty familiar with customizing vim, but there's always
      something I've missed!). Or guidance on hacking vim if Bram would
      be likely to accept changes along these lines.

      Sorry for the long message.

      Andrew
    • Benji Fisher
      ... [snip] ... [snip] ... Use WinLeave autocommands to set a variable (script-local, perhaps) and WinEnter autocommands to set the cusor. [snip] ... Example?
      Message 2 of 12 , Feb 3 5:23 PM
      • 0 Attachment
        Andrew Pimlott wrote:
        > I have for a long time wanted vim to be better at editing
        > "comma-separated value" and other column-oriented files. When I saw
        > the vertical split and other features added to vim 6, I thought I
        > might be able to hack it together, but there are some pieces
        > missing. So I'll describe the idea and suggest what I think is
        > needed to implement it.
        [snip]
        >
        > Note that I'm not asking for spreadsheet features in vim. I just
        > want to treat my CSV files as columns of text.
        >
        > The basic idea is to split the file into columns in some automated
        > way and put each column into a vertically-split window (possibly
        > with a thin horizontal header window at the top). Editing should
        > mostly work as normal, except
        [snip]
        >
        > Of course, I'm willing to settle for some reasonable subset of this
        > functionality.
        >
        > What I have:
        >
        > * scroll-binding keeps the windows lined-up, but it doesn't keep the
        > cursor on the same row in every window. This seems like a useful
        > addition. Actually, I don't see any reason not to do this by
        > default when scrollbind is set, because even now the cursor is
        > moved in the non-current windows when it would go off-screen.
        > Keeping it on the same (relative) row in all windows would be more
        > consistent.
        >
        > I think this would have to be done within vim itself. The only
        > alternative would be an autocommand for cursor motions. Is that
        > practical?

        Use WinLeave autocommands to set a variable (script-local,
        perhaps) and WinEnter autocommands to set the cusor.

        [snip]
        > * Operations that should prefer the current column then fall back to
        > other columns would probably have to be done within vim. This is
        > already possible for completion, but not AFAICT for searching.
        > This is probably a relatively low priority, though.

        Example?

        > * Data common to all columns would have to be done within vim. This
        > seems to require pretty intrusive changes, but I really don't
        > think I would be able to stand not being able to undo across
        > columns. vim would probably have to learn the notion of several
        > buffers being one "logical buffer". Of course, once you do this,
        > most of the other changes above fall out naturally.

        What's wrong with :windo undo ? Provide a user command, :map, or
        :menu for it if you like.

        > * Visual selection across columns is just me musing.
        >
        > Any suggestions or recommendations about how I should go about this
        > are welcome. In particular, ways to do things without changing vim
        > itself (I'm pretty familiar with customizing vim, but there's always
        > something I've missed!). Or guidance on hacking vim if Bram would
        > be likely to accept changes along these lines.
        >
        > Sorry for the long message.

        I think it is easier to respond to a message with a couple of more
        specific questions. Good luck.

        --Benji Fisher
      • Charles E. Campbell
        ... If they re comma-separated, you can use Align http://vim.sourceforge.net/scripts/script.php?script_id=294 to line em up. ... You can use the status line
        Message 3 of 12 , Feb 4 12:37 PM
        • 0 Attachment
          On Mon, Feb 03, 2003 at 03:46:48PM -0500, Andrew Pimlott wrote:
          > Of course, you can edit CSV and similar files with vim, but there
          > are some drawbacks:
          >
          > - The columns aren't lined up visually

          If they're comma-separated, you can use Align

          http://vim.sourceforge.net/scripts/script.php?script_id=294

          to line 'em up.

          > - You can't easily tell which column you're on

          You can use the status line for this. Write a function to convert
          screen column into logical column after using Align.

          > - You can't limit operations to a column

          Check out http://vim.sourceforge.net/tips/tip.php?tip_id=63,
          which allows more or less arbitrary ex commands to restrict
          their depradations to just a visual-block.

          > - columns should stay lined-up relative to each other

          Probably could use CursorHold to call Align occasionally.
          Alternatives: set up a map and let the user line things
          up occasionally.

          > - it should be easy to move between columns

          This is a question of setting up a map or two. f, F, ?

          > - some operations (searching, completing) should look first at the
          > current column, then on the other columns

          This operation could be done via a map and functions. Re-use
          the information from above which "knows" where columns begin/end.
          Use a function call for the search; change the expression to
          include column information (ie. use \%23c.*pattern.*\%46c to handle
          searches between columns 23-46).

          > * Splitting files into columns

          That's interesting -- I see you're working with a multiple buffer/window
          approach. Your column count will be limited by screen space.

          Another idea: you could use a three-window approach:

          all-columns-left all-columns-right
          of-current-column current-column of-current-column

          with obvious modification for the first/last columns.

          > * Performing operations on all columns can be done to some extent
          > with mappings.

          windo/bufdo are good for this.

          Regards,
          C Campbell

          --
          Charles E Campbell, Jr, PhD _ __ __
          Goddard Space Flight Center / /_/\_\_/ /
          cec@... /_/ \/_//_/
          PGP public key: http://www.erols.com/astronaut/pgp.html
        • Andrew Pimlott
          ... Hey, that s pretty slick! I just downloaded it and am playing around. But there are some major drawbacks for what I am trying to do: It is not real-time,
          Message 4 of 12 , Feb 4 1:17 PM
          • 0 Attachment
            On Tue, Feb 04, 2003 at 03:37:48PM -0500, Charles E. Campbell wrote:
            > On Mon, Feb 03, 2003 at 03:46:48PM -0500, Andrew Pimlott wrote:
            > > Of course, you can edit CSV and similar files with vim, but there
            > > are some drawbacks:
            > >
            > > - The columns aren't lined up visually
            >
            > If they're comma-separated, you can use Align

            Hey, that's pretty slick! I just downloaded it and am playing
            around.

            But there are some major drawbacks for what I am trying to do: It
            is not real-time, it does not gracefully handle a "cell" with a long
            entry, and it does not help with escaping (ie, a cell that contains
            the delimiter).

            Honestly, if I were happy with this approach, I would just write a
            perl script to do all my formatting and pipe through that.

            > > - You can't limit operations to a column
            >
            > Check out http://vim.sourceforge.net/tips/tip.php?tip_id=63,
            > which allows more or less arbitrary ex commands to restrict
            > their depradations to just a visual-block.

            Heh, cool. I was just waiting for this to show up in vim, because
            the docs say "In a future release ":" may work on partial lines.".
            :-)

            > > - columns should stay lined-up relative to each other
            >
            > Probably could use CursorHold to call Align occasionally.

            Maybe I'll try that, but it seems a bit too klugy.

            > > - some operations (searching, completing) should look first at the
            > > current column, then on the other columns
            >
            > This operation could be done via a map and functions. Re-use
            > the information from above which "knows" where columns begin/end.
            > Use a function call for the search; change the expression to
            > include column information (ie. use \%23c.*pattern.*\%46c to handle
            > searches between columns 23-46).

            This would kill incremental searching, which would be a noticable
            regression. I find it hard to live without that now. But the need
            to search only one column should be relatively uncommon, so maybe I
            can live without incremental search in that case.

            > > * Splitting files into columns
            >
            > That's interesting -- I see you're working with a multiple buffer/window
            > approach. Your column count will be limited by screen space.

            That is a tiny disadvantage to me. The big advantage is that a wide
            column doesn't have to force everything else off the screen.

            I thought when I started out about keeping everything in one window,
            but there seem to be so many little problems with this. I convinced
            myself that multi-window was the right answer since it keeps the
            columns truly independent. As it stands, this approach is mostly
            working, and I think I can iron out all of the kinks except one:
            undo. It really sucks to have to move from column to column to undo
            a change that affected all columns (eg, deleting a row). But maybe
            "windo normal undo" will prove good enough.

            > Another idea: you could use a three-window approach:
            >
            > all-columns-left all-columns-right
            > of-current-column current-column of-current-column
            >
            > with obvious modification for the first/last columns.

            ??? Not sure what advantages you see here. Is it that this makes
            better use of screen space? I only expect to have five or six
            columns, at most, so I don't expect that to be a big problem.

            Thanks for your ideas (and your code).

            Andrew
          • Bram Moolenaar
            ... This sounds like very specialized editing, and not too much related to generic text editing. Thus I don t think this should be added to the Vim code
            Message 5 of 12 , Feb 4 2:15 PM
            • 0 Attachment
              Andrew Pimlott wrote:

              > Any suggestions or recommendations about how I should go about this
              > are welcome. In particular, ways to do things without changing vim
              > itself (I'm pretty familiar with customizing vim, but there's always
              > something I've missed!). Or guidance on hacking vim if Bram would
              > be likely to accept changes along these lines.

              This sounds like very specialized editing, and not too much related to
              generic text editing. Thus I don't think this should be added to the
              Vim code itself.

              I'm sure you can do most things with Vim scripts (plugins, mappings,
              functions, etc.). If that fails, try using Python or Perl.

              Perhaps you can already do quite a lot if you make sure there is a
              single TAB separating the columns and set 'tabstop' to a large value.
              Autocommands could convert the file after reading and before writing.

              --
              hundred-and-one symptoms of being an internet addict:
              120. You ask a friend, "What's that big shiny thing?" He says, "It's the sun."

              /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
              /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\
              \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
              \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html ///
            • Andrew Pimlott
              ... That works great, thanks. I was stuck thinking that the cursor update had to be real-time in all windows, but it only really matters when you switch.
              Message 6 of 12 , Feb 4 7:39 PM
              • 0 Attachment
                On Mon, Feb 03, 2003 at 08:23:23PM -0500, Benji Fisher wrote:
                > Andrew Pimlott wrote:
                > >* scroll-binding keeps the windows lined-up, but it doesn't keep the
                > > cursor on the same row in every window. This seems like a useful
                > > addition. Actually, I don't see any reason not to do this by
                > > default when scrollbind is set, because even now the cursor is
                > > moved in the non-current windows when it would go off-screen.
                > > Keeping it on the same (relative) row in all windows would be more
                > > consistent.
                > >
                > > I think this would have to be done within vim itself. The only
                > > alternative would be an autocommand for cursor motions. Is that
                > > practical?
                >
                > Use WinLeave autocommands to set a variable (script-local,
                > perhaps) and WinEnter autocommands to set the cusor.

                That works great, thanks. I was stuck thinking that the cursor
                update had to be real-time in all windows, but it only really
                matters when you switch. Duh.

                > [snip]
                > >* Operations that should prefer the current column then fall back to
                > > other columns would probably have to be done within vim. This is
                > > already possible for completion, but not AFAICT for searching.
                > > This is probably a relatively low priority, though.
                >
                > Example?

                Mostly searching, I guess. I haven't tried, but I think it would be
                hard to do this well with a function. You want incremental
                searching, highlighting, next/previous, etc to work as usual. But
                perhaps I'll try emulating this and see where I get.

                > >* Data common to all columns would have to be done within vim. This
                > > seems to require pretty intrusive changes, but I really don't
                > > think I would be able to stand not being able to undo across
                > > columns. vim would probably have to learn the notion of several
                > > buffers being one "logical buffer". Of course, once you do this,
                > > most of the other changes above fall out naturally.
                >
                > What's wrong with :windo undo ?

                I want "u" to undo one change at a time, in the opposite order of
                the changes, regardless of which column the changes are in.
                Actually, what would be even better is if I map "dd" to a function
                that deletes a line in every column, then a single "u" should undo
                the whole thing.

                BTW, undo is the only difficult case of shared data I can think of.
                Jumps are already cross-buffer, and marks can be global. But is
                there any way, when going to a jump or mark in another buffer, to
                switch to an existing window containing that buffer, rather than
                loading the buffer into the current window?

                > I think it is easier to respond to a message with a couple of more
                > specific questions. Good luck.

                You're right, as I realize now on rereading it. Hopefully this
                message is a start. Based on the progress I'm making, I think that
                undo and search across buffers are the biggest issues.

                Andrew
              • Andrew Pimlott
                ... Understood. However, can I cast a couple of my ideas in terms of general editing? - Global undo. Say you ve been making related changes to several files
                Message 7 of 12 , Feb 6 5:33 PM
                • 0 Attachment
                  On Tue, Feb 04, 2003 at 11:15:41PM +0100, Bram Moolenaar wrote:
                  > This sounds like very specialized editing, and not too much related to
                  > generic text editing. Thus I don't think this should be added to the
                  > Vim code itself.

                  Understood. However, can I cast a couple of my ideas in terms of
                  general editing?

                  - Global undo. Say you've been making related changes to several
                  files and you want to undo all of the recent changes. Global undo
                  would undo changes in any buffers, in the reverse order in which
                  they were made. The implementation would (hopefully) be as simple
                  as keeping all undo entries in a global list.

                  - I hoped that 'switchbuf' (which I just discovered) would take
                  care of my next wish, but it doesn't. I basically want the
                  switchbuf option honored for _all_ commands that switch buffers.
                  Eg, tags, marks, jumps. Is there any reason it only applies to a
                  few commands? (I may be able to simulate this with BufEnter, but
                  it seems like switchbuf ought to do it.)

                  Do either of these sound useful to you?

                  > Perhaps you can already do quite a lot if you make sure there is a
                  > single TAB separating the columns and set 'tabstop' to a large value.
                  > Autocommands could convert the file after reading and before writing.

                  Some "cells" have relatively long entries, so this would be a
                  problem. For this reason, I'm pretty set on the approach of putting
                  each column in its own window, using vertical splitting.

                  Thanks,
                  Andrew
                • Benji Fisher
                  ... [snip] ... Here is another option, in case you are willing to consider rewriting your script entirely. Use BufEnter and BufWrite autocommands toconvert
                  Message 8 of 12 , Feb 7 8:29 AM
                  • 0 Attachment
                    Andrew Pimlott wrote:
                    > On Tue, Feb 04, 2003 at 11:15:41PM +0100, Bram Moolenaar wrote:
                    >
                    [snip]
                    >>Perhaps you can already do quite a lot if you make sure there is a
                    >>single TAB separating the columns and set 'tabstop' to a large value.
                    >>Autocommands could convert the file after reading and before writing.
                    >
                    > Some "cells" have relatively long entries, so this would be a
                    > problem. For this reason, I'm pretty set on the approach of putting
                    > each column in its own window, using vertical splitting.

                    Here is another option, in case you are willing to consider
                    rewriting your script entirely. Use BufEnter and BufWrite autocommands
                    toconvert between the usual format

                    ==comma-separated fields==
                    a,b,c,d
                    1,2,3,4
                    ==========================

                    and a single buffer that is only one column wide:

                    ==strung out for editing==
                    a
                    1

                    b
                    2

                    c
                    3

                    d
                    4

                    ==========================

                    Then, open several windows on this single buffer, one for each column.
                    (Actually, this is not necessary: if there are a lot of columns, you
                    can open windows on a few of them.) I think it is possible to use
                    'scrollbind' to keep then in sync. You might want to pad with empty
                    lines, so that you never see the top of one column in the bottom of
                    another column's window.

                    Advantages:

                    * You automatically get "global" undo since it is all a single buffer.
                    * Fewer temporary files

                    Disadvantages:

                    * Jumps (e.g., when doing an undo) keep you in the same window, even if
                    they go to something displayed in another window. I do not know of an
                    option to change this, but you can always try a mapping...

                    I leave it to you to extend these lists.

                    HTH --Benji Fisher
                  • Andrew Pimlott
                    ... This was trivial to implement. There are five uses of buflist_getfile: jumps, marks, CTRL-^, quickfix, and tags. Quickfix was the only one that passed
                    Message 9 of 12 , Feb 8 11:17 AM
                    • 0 Attachment
                      On Thu, Feb 06, 2003 at 08:33:27PM -0500, Andrew Pimlott wrote:
                      > - I hoped that 'switchbuf' (which I just discovered) would take
                      > care of my next wish, but it doesn't. I basically want the
                      > switchbuf option honored for _all_ commands that switch buffers.
                      > Eg, tags, marks, jumps. Is there any reason it only applies to a
                      > few commands? (I may be able to simulate this with BufEnter, but
                      > it seems like switchbuf ought to do it.)

                      This was trivial to implement. There are five uses of
                      buflist_getfile: jumps, marks, CTRL-^, quickfix, and tags. Quickfix
                      was the only one that passed GETF_SWITCH to honor switchbuf. As far
                      as I can see, there is no reason they shouldn't all honor switchbuf,
                      so I changed them all. Of course, if you were to do that, you may
                      as well not have the flag in the first place.

                      I've used this for a bit, and it feels intuitive and consistent, and
                      no problems have popped up. So I would vote for getting rid of
                      GETF_SWITCH.

                      Andrew

                      --- src/mark.c.orig 2003-02-08 13:04:14.000000000 -0500
                      +++ src/mark.c 2003-02-08 13:01:42.000000000 -0500
                      @@ -208,7 +208,7 @@
                      continue;
                      }
                      if (buflist_getfile(jmp->fmark.fnum, jmp->fmark.mark.lnum,
                      - 0, FALSE) == FAIL)
                      + GETF_SWITCH, FALSE) == FAIL)
                      return (pos_T *)NULL;
                      /* Set lnum again, autocommands my have changed it */
                      curwin->w_cursor = jmp->fmark.mark;
                      @@ -344,7 +344,7 @@
                      && changefile && namedfm[c].fmark.fnum)
                      {
                      if (buflist_getfile(namedfm[c].fmark.fnum,
                      - (linenr_T)1, GETF_SETMARK, FALSE) == OK)
                      + (linenr_T)1, GETF_SETMARK|GETF_SWITCH, FALSE) == OK)
                      {
                      /* Set the lnum now, autocommands could have changed it */
                      curwin->w_cursor = namedfm[c].fmark.mark;
                      --- src/normal.c.orig 2003-02-08 13:04:00.000000000 -0500
                      +++ src/normal.c 2003-02-08 13:01:59.000000000 -0500
                      @@ -4549,7 +4549,7 @@
                      {
                      if (!checkclearopq(cap->oap))
                      (void)buflist_getfile((int)cap->count0, (linenr_T)0,
                      - GETF_SETMARK|GETF_ALT, FALSE);
                      + GETF_SETMARK|GETF_ALT|GETF_SWITCH, FALSE);
                      }

                      /*
                      --- src/tag.c.orig 2003-02-08 13:03:40.000000000 -0500
                      +++ src/tag.c 2003-02-08 13:02:29.000000000 -0500
                      @@ -302,7 +302,7 @@
                      * file was changed) keep original position in tag stack.
                      */
                      if (buflist_getfile(saved_fmark.fnum, saved_fmark.mark.lnum,
                      - GETF_SETMARK, forceit) == FAIL)
                      + GETF_SETMARK|GETF_SWITCH, forceit) == FAIL)
                      {
                      tagstackidx = oldtagstackidx; /* back to old posn */
                      goto end_do_tag;
                    • Andrew Pimlott
                      ... I implemented this. The appended patch is a first draft. Feedback solicited. I believe it is stable, so please try it and tell me if you find it useful!
                      Message 10 of 12 , Feb 8 12:44 PM
                      • 0 Attachment
                        On Thu, Feb 06, 2003 at 08:33:27PM -0500, Andrew Pimlott wrote:
                        > - Global undo. Say you've been making related changes to several
                        > files and you want to undo all of the recent changes. Global undo
                        > would undo changes in any buffers, in the reverse order in which
                        > they were made. The implementation would (hopefully) be as simple
                        > as keeping all undo entries in a global list.

                        I implemented this. The appended patch is a first draft. Feedback
                        solicited. I believe it is stable, so please try it and tell me if
                        you find it useful! I'll admit that I have an ulterior motive for
                        this feature which I described in earlier mails; however, it has
                        come in handy for "normal" editing as well. For example, when
                        editing several files, you sometimes want to review all the recent
                        changes, for which you just can global undo a bit, then global redo
                        back to where you were. But enough hype!

                        This is the "strong" version of global undo. Global undo and redo
                        are always available (unless everything has been undone or redone),
                        and always undo the latest/redo the earliest change available. The
                        order of undos is the order in which the changes were originally
                        made, and is not affected by subsequent undoing and redoing. This
                        means that global undo and redo are not always inverses; ie a global
                        undo followed by a global redo might redo a different (earlier)
                        change. This is potentially confusing as well as potential useful.
                        There are some variations on this I could try if it turn out to be
                        confusing. However, in the common cases, it works as you expect.

                        You may want to set 'hidden' in conjuction with this feature.
                        Otherwise, you will lose your undo list for a buffer when you leave
                        it. (I'd like to rave about the 'hidden' option in general. I
                        started using it less than a week ago, and it has already changed
                        the way I edit. It's not about having hidden modified buffers--it's
                        about remembering undo lists!)

                        When global undo has to switch buffers, it honors switchbuf (cf my
                        last mail). So you can use it without 'hidden' by keeping your
                        files open in different windows, and global undo will switch among
                        them.

                        I bound the commands only to :gundo and :gredo (aren't they Tolkien
                        characters?). There should probably be some default key bindings,
                        but I haven't settled on any. Suggestions for better bindings
                        welcome.

                        Here are some notes on the implementation.

                        u_headers are now doubly-linked globally, as well as per buffer. They also
                        have three new attributes:

                        - a buffer number, since we need it when walking the global list
                        - the global sequence number of the undo, incremented for every new undo
                        - a UH_UNDONE flag, indicating that it has been undone (and not redone)

                        There are also four new globals:

                        - a pointer to the newest global undo header
                        - a pointer to the current global undo header
                        - a pointer to the current global redo header
                        - a counter for generating the above sequence numbers

                        (Note that the current global undo header points _at_ the next header to be
                        undone, not before it like the per-buffer current undo header.)

                        The algorithm for performing a global undo/redo is:

                        1. Move the current global undo/redo header forward/backward across the
                        global header list, returning if it becomes null.
                        2. Stop at the first header that has UH_UNDONE cleared/set.
                        3. Perform the undo/redo on that header.
                        4. Move the current global undo/redo header one more step forward/backward.

                        The algorithm for peforming an undo/redo has the following new steps:

                        1. Set/clear the UH_UNDONE flag.
                        2. If the current global redo/undo header is null, make the header the
                        current global redo/undo header.
                        3. If the sequence number of the header is less/greater than the sequence
                        number of the current global redo/undo header, make the header the
                        current global redo/undo header.

                        The algorithm for adding an undo header has the following new steps:
                        1. Set the sequence number from the counter, then increment the counter.
                        2. Link the header to the newest global undo header (which may be null).
                        3. Make the header the newest global undo header.
                        4. Make the header the current global undo header.

                        The algorithm for freeing an undo header has the following new steps:
                        1. Remove the header from the global list.
                        2. If it was the current global undo/redo header, make the next/previous
                        global header the current global undo/redo header.

                        The algorithm for unloading a buffer has the following new steps:

                        1. Walk the buffer's u_header list.
                        2. Free every header as above.

                        Most of this fit into place quite easily, which is a credit to vim's
                        clean code. One thing I wasn't sure of was whether to change
                        u_blockfree like I did, or add another function to make more clear
                        the side-effects. There is also some debugging code, which can be
                        removed.

                        Andrew

                        --- src/structs.h.orig 2003-02-07 19:37:19.000000000 -0500
                        +++ src/structs.h 2003-02-08 14:27:47.000000000 -0500
                        @@ -231,6 +231,10 @@ struct u_header
                        {
                        u_header_T *uh_next; /* pointer to next header in list */
                        u_header_T *uh_prev; /* pointer to previous header in list */
                        + u_header_T *uh_g_next; /* pointer to next header in global list */
                        + u_header_T *uh_g_prev; /* pointer to previous header in global list */
                        + int uh_g_seqno; /* global undo sequence number */
                        + int *uh_fnum; /* file number (needed for global list) */
                        u_entry_T *uh_entry; /* pointer to first entry */
                        u_entry_T *uh_getbot_entry; /* pointer to where ue_bot must be set */
                        pos_T uh_cursor; /* cursor position before saving */
                        @@ -244,6 +248,7 @@ struct u_header
                        /* values for uh_flags */
                        #define UH_CHANGED 0x01 /* b_changed flag before undo/after redo */
                        #define UH_EMPTYBUF 0x02 /* buffer was empty */
                        +#define UH_UNDONE 0x04 /* set when undone, cleared when redone */

                        /*
                        * stuctures used in undo.c
                        --- src/globals.h.orig 2003-02-07 19:37:10.000000000 -0500
                        +++ src/globals.h 2003-02-07 22:08:33.000000000 -0500
                        @@ -394,6 +394,11 @@ EXTERN alist_T global_alist; /* global a
                        EXTERN int arg_had_last INIT(= FALSE); /* accessed last file in
                        global_alist */

                        +EXTERN u_header_T *g_u_newhead INIT(= NULL); /* newest global undo */
                        +EXTERN u_header_T *g_u_curhead INIT(= NULL); /* next global undo */
                        +EXTERN u_header_T *g_r_curhead INIT(= NULL); /* next global redo */
                        +EXTERN int g_u_count INIT(= 0); /* global count of undos */
                        +
                        EXTERN int ru_col; /* column for ruler */
                        #ifdef FEAT_STL_OPT
                        EXTERN int ru_wid; /* 'rulerfmt' width of ruler when non-zero */
                        --- src/ex_cmds.h.orig 2003-02-08 09:15:37.000000000 -0500
                        +++ src/ex_cmds.h 2003-02-08 09:15:44.000000000 -0500
                        @@ -370,8 +370,12 @@ EX(CMD_grep, "grep", ex_make,
                        BANG|NEEDARG|EXTRA|NOTRLCOM|TRLBAR|XFILE),
                        EX(CMD_grepadd, "grepadd", ex_make,
                        BANG|NEEDARG|EXTRA|NOTRLCOM|TRLBAR|XFILE),
                        +EX(CMD_gredo, "gredo", ex_gredo,
                        + TRLBAR|CMDWIN),
                        EX(CMD_gui, "gui", ex_gui,
                        BANG|FILES|EDITCMD|ARGOPT|TRLBAR|CMDWIN),
                        +EX(CMD_gundo, "gundo", ex_gundo,
                        + TRLBAR|CMDWIN),
                        EX(CMD_gvim, "gvim", ex_gui,
                        BANG|FILES|EDITCMD|ARGOPT|TRLBAR|CMDWIN),
                        EX(CMD_help, "help", ex_help,
                        --- src/ex_docmd.c.orig 2003-02-07 19:38:08.000000000 -0500
                        +++ src/ex_docmd.c 2003-02-08 02:46:40.000000000 -0500
                        @@ -266,6 +266,8 @@ static void ex_at __ARGS((exarg_T *eap))
                        static void ex_bang __ARGS((exarg_T *eap));
                        static void ex_undo __ARGS((exarg_T *eap));
                        static void ex_redo __ARGS((exarg_T *eap));
                        +static void ex_gundo __ARGS((exarg_T *eap));
                        +static void ex_gredo __ARGS((exarg_T *eap));
                        static void ex_redir __ARGS((exarg_T *eap));
                        static void ex_redraw __ARGS((exarg_T *eap));
                        static void close_redir __ARGS((void));
                        @@ -6439,6 +6441,28 @@ ex_redo(eap)
                        }

                        /*
                        + * ":gundo".
                        + */
                        +/*ARGSUSED*/
                        + static void
                        +ex_gundo(eap)
                        + exarg_T *eap;
                        +{
                        + u_g_undo(1);
                        +}
                        +
                        +/*
                        + * ":gredo".
                        + */
                        +/*ARGSUSED*/
                        + static void
                        +ex_gredo(eap)
                        + exarg_T *eap;
                        +{
                        + u_g_redo(1);
                        +}
                        +
                        +/*
                        * ":redir": start/stop redirection.
                        */
                        static void
                        --- src/undo.c.orig 2003-02-08 01:06:30.000000000 -0500
                        +++ src/undo.c 2003-02-08 14:29:03.000000000 -0500
                        @@ -51,6 +51,8 @@ static u_entry_T *u_get_headentry __ARGS
                        static void u_getbot __ARGS((void));
                        static int u_savecommon __ARGS((linenr_T, linenr_T, linenr_T));
                        static void u_doit __ARGS((int count));
                        +static void u_g_undoredo __ARGS((int undo, int count));
                        +static void u_undo_pre __ARGS((void));
                        static void u_undoredo __ARGS((void));
                        static void u_undo_end __ARGS((void));
                        static void u_freelist __ARGS((struct u_header *));
                        @@ -60,6 +62,7 @@ static char_u *u_blockalloc __ARGS((long
                        static void u_free_line __ARGS((char_u *, int keep));
                        static char_u *u_alloc_line __ARGS((unsigned));
                        static char_u *u_save_line __ARGS((linenr_T));
                        +static void u_g_check __ARGS((void));

                        static long u_newcount, u_oldcount;

                        @@ -223,6 +226,16 @@ u_savecommon(top, bot, newbot)
                        uhp->uh_next = curbuf->b_u_newhead;
                        if (curbuf->b_u_newhead != NULL)
                        curbuf->b_u_newhead->uh_prev = uhp;
                        +
                        + uhp->uh_g_prev = NULL;
                        + uhp->uh_g_next = g_u_newhead;
                        + if (g_u_newhead != NULL)
                        + g_u_newhead->uh_g_prev = uhp;
                        + g_u_newhead = uhp;
                        + uhp->uh_g_seqno = g_u_count++;
                        + g_u_curhead = uhp;
                        + uhp->uh_fnum = curbuf->b_fnum;
                        +
                        uhp->uh_entry = NULL;
                        uhp->uh_getbot_entry = NULL;
                        uhp->uh_cursor = curwin->w_cursor; /* save cursor pos. for undo */
                        @@ -436,23 +449,8 @@ u_redo(count)
                        u_doit(count)
                        int count;
                        {
                        - /* Don't allow changes when 'modifiable' is off. */
                        - if (!curbuf->b_p_ma)
                        - {
                        - EMSG(_(e_modifiable));
                        - return;
                        - }
                        -#ifdef HAVE_SANDBOX
                        - /* In the sandbox it's not allowed to change the text. */
                        - if (sandbox != 0)
                        - {
                        - EMSG(_(e_sandbox));
                        - return;
                        - }
                        -#endif
                        + u_undo_pre();

                        - u_newcount = 0;
                        - u_oldcount = 0;
                        while (count--)
                        {
                        if (undo_undoes)
                        @@ -490,6 +488,108 @@ u_doit(count)
                        }

                        /*
                        + * Global undo/redo.
                        + */
                        + void
                        +u_g_undo(count)
                        + int count;
                        +{
                        + u_g_undoredo(TRUE, count);
                        +}
                        + void
                        +u_g_redo(count)
                        + int count;
                        +{
                        + u_g_undoredo(FALSE, count);
                        +}
                        + static void
                        +u_g_undoredo(undo, count)
                        + int undo;
                        + int count;
                        +{
                        + u_undo_pre();
                        +
                        + while (count--) {
                        + u_header_T *uhp;
                        +
                        + /* scan the global undo list */
                        + if (undo) /* undo */
                        + while (g_u_curhead != NULL && g_u_curhead->uh_flags & UH_UNDONE)
                        + g_u_curhead = g_u_curhead->uh_g_next;
                        + else /* redo */
                        + while (g_r_curhead != NULL && ! (g_r_curhead->uh_flags & UH_UNDONE))
                        + g_r_curhead = g_r_curhead->uh_g_prev;
                        +
                        + uhp = undo ? g_u_curhead : g_r_curhead;
                        +
                        + if (uhp == NULL)
                        + {
                        + beep_flush();
                        + break;
                        + }
                        +
                        + if (uhp->uh_fnum != curbuf->b_fnum)
                        + if (buflist_getfile(uhp->uh_fnum,
                        + (linenr_T)1, GETF_SETMARK|GETF_SWITCH, FALSE) == FAIL)
                        + break;
                        +
                        + if (undo) /* undo */
                        + {
                        + if (curbuf->b_u_synced == FALSE)
                        + u_sync();
                        +
                        + if (curbuf->b_u_curhead == NULL) /* first undo */
                        + curbuf->b_u_curhead = curbuf->b_u_newhead;
                        + else /* get next undo */
                        + curbuf->b_u_curhead = curbuf->b_u_curhead->uh_next;
                        + }
                        +
                        + if (curbuf->b_u_curhead != uhp)
                        + {
                        + EMSG(_("E439: undo list corrupt"));
                        + break;
                        + }
                        +
                        + u_undoredo();
                        +
                        + if (undo) /* undo */
                        + g_u_curhead = g_u_curhead->uh_g_next;
                        + else { /* redo */
                        + g_r_curhead = g_r_curhead->uh_g_prev;
                        + curbuf->b_u_curhead = curbuf->b_u_curhead->uh_prev;
                        + }
                        +
                        + u_g_check();
                        + }
                        + u_undo_end();
                        +}
                        +
                        +/*
                        + * Called before an undo or redo.
                        + */
                        + static void
                        +u_undo_pre()
                        +{
                        + /* Don't allow changes when 'modifiable' is off. */
                        + if (!curbuf->b_p_ma)
                        + {
                        + EMSG(_(e_modifiable));
                        + return;
                        + }
                        +#ifdef HAVE_SANDBOX
                        + /* In the sandbox it's not allowed to change the text. */
                        + if (sandbox != 0)
                        + {
                        + EMSG(_(e_sandbox));
                        + return;
                        + }
                        +#endif
                        +
                        + u_newcount = 0;
                        + u_oldcount = 0;
                        +}
                        +
                        +/*
                        * u_undoredo: common code for undo and redo
                        *
                        * The lines in the file are replaced by the lines in the entry list at
                        @@ -515,7 +615,8 @@ u_undoredo()

                        old_flags = curbuf->b_u_curhead->uh_flags;
                        new_flags = (curbuf->b_changed ? UH_CHANGED : 0) +
                        - ((curbuf->b_ml.ml_flags & ML_EMPTY) ? UH_EMPTYBUF : 0);
                        + ((curbuf->b_ml.ml_flags & ML_EMPTY) ? UH_EMPTYBUF : 0) +
                        + ((old_flags & UH_UNDONE) ? 0 : UH_UNDONE);
                        setpcmark();

                        /*
                        @@ -689,6 +790,18 @@ u_undoredo()

                        /* Make sure the cursor is on an existing line and column. */
                        check_cursor();
                        +
                        + /* If we did an undo, maybe bump back the global redo header */
                        + if ( curbuf->b_u_curhead->uh_flags & UH_UNDONE &&
                        + (g_r_curhead == NULL ||
                        + g_r_curhead->uh_g_seqno > curbuf->b_u_curhead->uh_g_seqno))
                        + g_r_curhead = curbuf->b_u_curhead;
                        +
                        + /* If we did a redo, maybe bump back the global undo header */
                        + if ( ! (curbuf->b_u_curhead->uh_flags & UH_UNDONE) &&
                        + (g_u_curhead == NULL ||
                        + g_u_curhead->uh_g_seqno < curbuf->b_u_curhead->uh_g_seqno))
                        + g_u_curhead = curbuf->b_u_curhead;
                        }

                        /*
                        @@ -807,6 +920,27 @@ u_getbot()
                        }

                        /*
                        + * u_g_remove: remove an undo header from the global undo list
                        + */
                        + static void
                        +u_g_remove(uhp)
                        + struct u_header *uhp;
                        +{
                        + if (uhp->uh_g_next != NULL)
                        + uhp->uh_g_next->uh_g_prev = uhp->uh_g_prev;
                        +
                        + if (uhp->uh_g_prev == NULL)
                        + g_u_newhead = uhp->uh_g_next;
                        + else
                        + uhp->uh_g_prev->uh_g_next = uhp->uh_g_next;
                        +
                        + if (uhp == g_u_curhead)
                        + g_u_curhead = g_u_curhead->uh_g_next;
                        + if (uhp == g_r_curhead)
                        + g_r_curhead = g_r_curhead->uh_g_prev;
                        +}
                        +
                        +/*
                        * u_freelist: free one entry list and adjust the pointers
                        */
                        static void
                        @@ -834,6 +968,8 @@ u_freelist(uhp)
                        else
                        uhp->uh_prev->uh_next = uhp->uh_next;

                        + u_g_remove(uhp);
                        +
                        u_free_line((char_u *)uhp, FALSE);
                        --curbuf->b_u_numhead;
                        }
                        @@ -1049,8 +1185,14 @@ u_blockalloc(size)
                        u_blockfree(buf)
                        buf_T *buf;
                        {
                        + u_header_T *uhp;
                        mblock_T *p, *np;

                        + /* ??? is this the right place for this? Should it be its own function? */
                        + /* remove all undo headers from global undo list */
                        + for (uhp = buf->b_u_newhead; uhp != NULL; uhp = uhp->uh_next)
                        + u_g_remove(uhp);
                        +
                        for (p = buf->b_block_head.mb_next; p != NULL; p = np)
                        {
                        np = p->mb_next;
                        @@ -1330,3 +1472,50 @@ curbufIsChanged()
                        #endif
                        (curbuf->b_changed || file_ff_differs(curbuf));
                        }
                        +
                        +/*
                        + * Consistency check for global undo list. It's really simple: the undo
                        + * header has to catch all the undos, and the redo header has to catch all
                        + * the redos.
                        + */
                        + static void
                        +u_g_check()
                        +{
                        + u_header_T *uhp;
                        + int found_g_u = FALSE, found_g_r = FALSE;
                        +
                        + if (g_r_curhead == NULL)
                        + found_g_r = TRUE;
                        +
                        + for (uhp = g_u_newhead; uhp != NULL; uhp = uhp->uh_g_next) {
                        + if (uhp == g_u_curhead)
                        + found_g_u = TRUE;
                        + if ( uhp->uh_flags & UH_UNDONE && found_g_r ||
                        + ! (uhp->uh_flags & UH_UNDONE) && ! found_g_u)
                        + {
                        + EMSG(_("E439: undo list corrupt"));
                        + break;
                        + }
                        + if (uhp == g_r_curhead)
                        + found_g_r = TRUE;
                        + }
                        +}
                        +
                        +/*
                        + * Debugging.
                        + */
                        + static void
                        +dump_global_undo()
                        +{
                        + u_header_T *uhp;
                        +
                        + fprintf(stderr, " %9d\n", g_u_count);
                        + for (uhp = g_u_newhead; uhp != NULL; uhp = uhp->uh_g_next) {
                        + fprintf(stderr, "%c%c%c %9d %d\n",
                        + uhp == g_u_curhead ? 'v' : ' ',
                        + uhp == g_r_curhead ? '^' : ' ',
                        + uhp->uh_flags & UH_UNDONE ? '*' : ' ',
                        + uhp->uh_g_seqno,
                        + uhp->uh_fnum);
                        + }
                        +}
                      • Bram Moolenaar
                        ... I m quite sure users will want to have a choice if they want switchbuf to apply to all of these situations. Originally it was meant to be used for
                        Message 11 of 12 , Feb 9 1:52 PM
                        • 0 Attachment
                          Andrew Pimlott wrote:

                          > On Thu, Feb 06, 2003 at 08:33:27PM -0500, Andrew Pimlott wrote:
                          > > - I hoped that 'switchbuf' (which I just discovered) would take
                          > > care of my next wish, but it doesn't. I basically want the
                          > > switchbuf option honored for _all_ commands that switch buffers.
                          > > Eg, tags, marks, jumps. Is there any reason it only applies to a
                          > > few commands? (I may be able to simulate this with BufEnter, but
                          > > it seems like switchbuf ought to do it.)
                          >
                          > This was trivial to implement. There are five uses of
                          > buflist_getfile: jumps, marks, CTRL-^, quickfix, and tags. Quickfix
                          > was the only one that passed GETF_SWITCH to honor switchbuf. As far
                          > as I can see, there is no reason they shouldn't all honor switchbuf,
                          > so I changed them all. Of course, if you were to do that, you may
                          > as well not have the flag in the first place.
                          >
                          > I've used this for a bit, and it feels intuitive and consistent, and
                          > no problems have popped up. So I would vote for getting rid of
                          > GETF_SWITCH.

                          I'm quite sure users will want to have a choice if they want 'switchbuf'
                          to apply to all of these situations. Originally it was meant to be used
                          for quickfix commands only, so that an already open window was used for
                          ":cn". People who use it for that reason will be very surprised if
                          other commands suddenly behave differently.

                          Adding another value to 'switchbuf' would be possible, but it's not
                          obvious how to combine it with the existing "useopen" and "split"
                          without becoming incompatible with previous versions.

                          --
                          If Microsoft would build a car...
                          ... Occasionally your car would die on the freeway for no
                          reason. You would have to pull over to the side of the road,
                          close all of the car windows, shut it off, restart it, and
                          reopen the windows before you could continue. For some reason
                          you would simply accept this.

                          /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                          /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\
                          \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
                          \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html ///
                        • Bram Moolenaar
                          ... I had a bit of trouble understanding what this actually does. Looking at the code helped a lot. I think the main problem with this undo function is that
                          Message 12 of 12 , Feb 9 1:52 PM
                          • 0 Attachment
                            Andrew Pimlott wrote:

                            > On Thu, Feb 06, 2003 at 08:33:27PM -0500, Andrew Pimlott wrote:
                            > > - Global undo. Say you've been making related changes to several
                            > > files and you want to undo all of the recent changes. Global undo
                            > > would undo changes in any buffers, in the reverse order in which
                            > > they were made. The implementation would (hopefully) be as simple
                            > > as keeping all undo entries in a global list.
                            >
                            > I implemented this. The appended patch is a first draft. Feedback
                            > solicited. I believe it is stable, so please try it and tell me if
                            > you find it useful! I'll admit that I have an ulterior motive for
                            > this feature which I described in earlier mails; however, it has
                            > come in handy for "normal" editing as well. For example, when
                            > editing several files, you sometimes want to review all the recent
                            > changes, for which you just can global undo a bit, then global redo
                            > back to where you were. But enough hype!
                            >
                            > This is the "strong" version of global undo. Global undo and redo
                            > are always available (unless everything has been undone or redone),
                            > and always undo the latest/redo the earliest change available. The
                            > order of undos is the order in which the changes were originally
                            > made, and is not affected by subsequent undoing and redoing. This
                            > means that global undo and redo are not always inverses; ie a global
                            > undo followed by a global redo might redo a different (earlier)
                            > change. This is potentially confusing as well as potential useful.
                            > There are some variations on this I could try if it turn out to be
                            > confusing. However, in the common cases, it works as you expect.

                            I had a bit of trouble understanding what this actually does. Looking
                            at the code helped a lot.

                            I think the main problem with this undo function is that it only works
                            if the user really knows what he is doing. Once he closes a buffer (so
                            that it unloads) it no longer works as expected. Do we need to give an
                            error message then? Do we save the undo info in a file, so that we can
                            still perform the undo after unloading a buffer?

                            This raises more questions and will trigger many more feature requests.
                            I don't like it as it is.

                            > You may want to set 'hidden' in conjuction with this feature.
                            > Otherwise, you will lose your undo list for a buffer when you leave
                            > it. (I'd like to rave about the 'hidden' option in general. I
                            > started using it less than a week ago, and it has already changed
                            > the way I edit. It's not about having hidden modified buffers--it's
                            > about remembering undo lists!)

                            Which is exactly a reason why many people won't use it: It depends too
                            much on the user knowning what he is doing.

                            > I bound the commands only to :gundo and :gredo (aren't they Tolkien
                            > characters?). There should probably be some default key bindings,
                            > but I haven't settled on any. Suggestions for better bindings
                            > welcome.

                            Using the Ex commands takes quite a few keystrokes. The obvious "gu"
                            and "gr" are already taken. It's not easy to think of good
                            two-character commands.

                            --
                            If Microsoft would build a car...
                            ... The airbag system would ask "are you SURE?" before deploying.

                            /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                            /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\
                            \\\ Project leader for A-A-P -- http://www.A-A-P.org ///
                            \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html ///
                          Your message has been successfully submitted and would be delivered to recipients shortly.