Loading ...
Sorry, an error occurred while loading the content.

Re: capture() function to get output of command

Expand Messages
  • Christian Brabandt
    Hi Ingo! ... The patch throws out Warnings: eval.c: In function f_capture : eval.c:9305:16: warning: assignment from incompatible pointer type [enabled by
    Message 1 of 26 , May 2 7:29 AM
    • 0 Attachment
      Hi Ingo!

      On Do, 02 Mai 2013, Ingo Karkat wrote:

      > On 02-May-2013 15:34 +0200, Christian Brabandt wrote:
      >
      > > Hi mattn!
      > >
      > > On Mi, 01 Mai 2013, mattn wrote:
      > >
      > >> I wrote a patch to add new function 'capture()'.
      > >> When you want to get a list of all commands/functions/mappings with using vim script, probably, you'll do like follow.
      > >>
      > >> ---------------------------------
      > >> let funcs = ''
      > >> redir => funcs
      > >> silent! function
      > >> redir END
      > >> ---------------------------------
      > >>
      > >> I guess, most of vim script programmers want to get the result as expression. not command, want to get more easily.
      > >> If vim have 'capture()' which allow to getting those command line 'function', I guess this is very useful.
      > >>
      > >> For example, getting :Command list as array.
      > >> ---------------------------------
      > >> :echo map(split(capture("command"), "\n")[1:], 'split(v:val[4:])[0]')
      > >> ---------------------------------
      > >>
      > >> One more thing. vim doesn't support nest of :redir.
      > >> ---------------------------------
      > >> function! s:foo()
      > >> let a = ''
      > >> redir => a
      > >> silent echo "foo"
      > >> redir END
      > >> return a
      > >> endfunction
      > >>
      > >> function! s:bar()
      > >> let a = ''
      > >> redir => a
      > >> call s:foo()
      > >> redir END
      > >> return a
      > >> endfunction
      > >>
      > >> echo s:bar()
      > >> ---------------------------------
      > >>
      > >> This 'echo s:bar()' doesn't show anything. But capture() can do it.
      > >
      > > Perhaps we should first fix redir before we introduce another similar
      > > function?
      >
      > That would work, too. On the other hand, here's an idea from a
      > practitioner that might justify a separate function: Almost always, the
      > captured text is split() into a list of lines (mattn's example does
      > this, too). Why not have capture() return a list of lines in the first
      > place?! We obviously cannot change the original :redir interface, but a
      > new function could offer this and spare all plugins from doing that.

      The patch throws out Warnings:

      eval.c: In function 'f_capture':
      eval.c:9305:16: warning: assignment from incompatible pointer type [enabled by default]
      gcc -c -I. -Iproto -DHAVE_CONFIG_H -I/usr/local/include -g -DDEBUG -Wall -Wshadow -Wmissing-prototypes -Wmaybe-uninitialized -o objects/main.o main.c
      In file included from vim.h:1965:0,
      from main.c:11:
      globals.h:1098:18: warning: 'capture_ga' initialized and declared 'extern' [enabled by default]


      This patch seems to fix those warnings:
      iff --git a/src/eval.c b/src/eval.c
      --- a/src/eval.c
      +++ b/src/eval.c
      @@ -9302,7 +9302,7 @@
      if (check_secure())
      return;

      - capture_ga = alloc(sizeof(garray_T));
      + capture_ga = (garray_T *) alloc(sizeof(garray_T));
      ga_init2(capture_ga, (int)sizeof(char), 80);

      ++msg_silent;
      diff --git a/src/globals.h b/src/globals.h
      --- a/src/globals.h
      +++ b/src/globals.h
      @@ -1095,7 +1095,11 @@
      EXTERN int redir_reg INIT(= 0); /* message redirection register */
      EXTERN int redir_vname INIT(= 0); /* message redirection variable */
      #endif
      -extern garray_T *capture_ga INIT(= NULL); /* capture() buffer */
      +EXTERN garray_T *capture_ga /* capture() buffer */
      +# ifdef DO_INIT
      + = NULL
      +# endif
      + ;

      I haven't played yet much with the capture() function, so I am not sure,
      this works recursively.

      It looks like a useful addition, but I am not sure, whether a new
      function is the right way to go, instead of fixing redir.

      Mit freundlichen Grüßen
      Christian
      --
      Auf der Verpackung steht: Win95, NT oder besser, folglich ist Linux
      eine unterstützte Plattform.

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • Ingo Karkat
      ... I fully agree, it would be better if :redir could be fixed; even with my proposed inclusion of split()ing into lines, the additional benefit is low, and it
      Message 2 of 26 , May 2 8:01 AM
      • 0 Attachment
        On 02-May-2013 16:29 +0200, Christian Brabandt wrote:

        > [100 lines deleted]
        >
        > It looks like a useful addition, but I am not sure, whether a new
        > function is the right way to go, instead of fixing redir.

        I fully agree, it would be better if :redir could be fixed; even with my
        proposed inclusion of split()ing into lines, the additional benefit is
        low, and it will take a long time until plugins can fully benefit from
        that function without a compatibility wrapper.

        Because I'm still mostly ignorant of the Vim source code, I had assumed
        the patch author has found fixing :redir too difficult (as it also /
        mostly handles file and register targets where there indeed will be
        issues with overriding); I hope mattn can shed some light on that.

        -- regards, ingo

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
      • mattn
        ... Ah, thank you. -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more
        Message 3 of 26 , May 6 8:50 PM
        • 0 Attachment
          On Thursday, May 2, 2013 11:29:48 PM UTC+9, Christian Brabandt wrote:
          > Hi Ingo!
          >
          >
          >
          > On Do, 02 Mai 2013, Ingo Karkat wrote:
          >
          >
          >
          > > On 02-May-2013 15:34 +0200, Christian Brabandt wrote:
          >
          > >
          >
          > > > Hi mattn!
          >
          > > >
          >
          > > > On Mi, 01 Mai 2013, mattn wrote:
          >
          > > >
          >
          > > >> I wrote a patch to add new function 'capture()'.
          >
          > > >> When you want to get a list of all commands/functions/mappings with using vim script, probably, you'll do like follow.
          >
          > > >>
          >
          > > >> ---------------------------------
          >
          > > >> let funcs = ''
          >
          > > >> redir => funcs
          >
          > > >> silent! function
          >
          > > >> redir END
          >
          > > >> ---------------------------------
          >
          > > >>
          >
          > > >> I guess, most of vim script programmers want to get the result as expression. not command, want to get more easily.
          >
          > > >> If vim have 'capture()' which allow to getting those command line 'function', I guess this is very useful.
          >
          > > >>
          >
          > > >> For example, getting :Command list as array.
          >
          > > >> ---------------------------------
          >
          > > >> :echo map(split(capture("command"), "\n")[1:], 'split(v:val[4:])[0]')
          >
          > > >> ---------------------------------
          >
          > > >>
          >
          > > >> One more thing. vim doesn't support nest of :redir.
          >
          > > >> ---------------------------------
          >
          > > >> function! s:foo()
          >
          > > >> let a = ''
          >
          > > >> redir => a
          >
          > > >> silent echo "foo"
          >
          > > >> redir END
          >
          > > >> return a
          >
          > > >> endfunction
          >
          > > >>
          >
          > > >> function! s:bar()
          >
          > > >> let a = ''
          >
          > > >> redir => a
          >
          > > >> call s:foo()
          >
          > > >> redir END
          >
          > > >> return a
          >
          > > >> endfunction
          >
          > > >>
          >
          > > >> echo s:bar()
          >
          > > >> ---------------------------------
          >
          > > >>
          >
          > > >> This 'echo s:bar()' doesn't show anything. But capture() can do it.
          >
          > > >
          >
          > > > Perhaps we should first fix redir before we introduce another similar
          >
          > > > function?
          >
          > >
          >
          > > That would work, too. On the other hand, here's an idea from a
          >
          > > practitioner that might justify a separate function: Almost always, the
          >
          > > captured text is split() into a list of lines (mattn's example does
          >
          > > this, too). Why not have capture() return a list of lines in the first
          >
          > > place?! We obviously cannot change the original :redir interface, but a
          >
          > > new function could offer this and spare all plugins from doing that.
          >
          >
          >
          > The patch throws out Warnings:
          >
          >
          >
          > eval.c: In function 'f_capture':
          >
          > eval.c:9305:16: warning: assignment from incompatible pointer type [enabled by default]
          >
          > gcc -c -I. -Iproto -DHAVE_CONFIG_H -I/usr/local/include -g -DDEBUG -Wall -Wshadow -Wmissing-prototypes -Wmaybe-uninitialized -o objects/main.o main.c
          >
          > In file included from vim.h:1965:0,
          >
          > from main.c:11:
          >
          > globals.h:1098:18: warning: 'capture_ga' initialized and declared 'extern' [enabled by default]
          >
          >
          >
          >
          >
          > This patch seems to fix those warnings:
          >
          > iff --git a/src/eval.c b/src/eval.c
          >
          > --- a/src/eval.c
          >
          > +++ b/src/eval.c
          >
          > @@ -9302,7 +9302,7 @@
          >
          > if (check_secure())
          >
          > return;
          >
          >
          >
          > - capture_ga = alloc(sizeof(garray_T));
          >
          > + capture_ga = (garray_T *) alloc(sizeof(garray_T));
          >
          > ga_init2(capture_ga, (int)sizeof(char), 80);
          >
          >
          >
          > ++msg_silent;
          >
          > diff --git a/src/globals.h b/src/globals.h
          >
          > --- a/src/globals.h
          >
          > +++ b/src/globals.h
          >
          > @@ -1095,7 +1095,11 @@
          >
          > EXTERN int redir_reg INIT(= 0); /* message redirection register */
          >
          > EXTERN int redir_vname INIT(= 0); /* message redirection variable */
          >
          > #endif
          >
          > -extern garray_T *capture_ga INIT(= NULL); /* capture() buffer */
          >
          > +EXTERN garray_T *capture_ga /* capture() buffer */
          >
          > +# ifdef DO_INIT
          >
          > + = NULL
          >
          > +# endif
          >
          > + ;
          >
          >
          >
          > I haven't played yet much with the capture() function, so I am not sure,
          >
          > this works recursively.
          >
          >
          >
          > It looks like a useful addition, but I am not sure, whether a new
          >
          > function is the right way to go, instead of fixing redir.
          >
          >
          >
          > Mit freundlichen Gr��en
          >
          > Christian
          >
          > --
          >
          > Auf der Verpackung steht: Win95, NT oder besser, folglich ist Linux
          >
          > eine unterst�tzte Plattform.

          Ah, thank you.

          --
          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_dev" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • mattn
          ... The functions in message.c write message with color attributes. So if splitting per line, it must get whole contents at the first. I guess someone want to
          Message 4 of 26 , May 6 8:57 PM
          • 0 Attachment
            > I fully agree, it would be better if :redir could be fixed; even with my
            > proposed inclusion of split()ing into lines, the additional benefit is
            > low, and it will take a long time until plugins can fully benefit from
            > that function without a compatibility wrapper.
            >
            > Because I'm still mostly ignorant of the Vim source code, I had assumed
            > the patch author has found fixing :redir too difficult (as it also /
            > mostly handles file and register targets where there indeed will be
            > issues with overriding); I hope mattn can shed some light on that.

            The functions in message.c write message with color attributes. So if splitting per line, it must get whole contents at the first.
            I guess someone want to get the contents as text which contains \n, but someone want to get the contents as array of lines. So I want to keep the contents as text.

            Thanks.

            --
            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php

            ---
            You received this message because you are subscribed to the Google Groups "vim_dev" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
          • Ingo Karkat
            ... Well, okay; right now, if split() is called separately, that works out the same. ... My point is that I ve *never* seen that use case so far in plugins;
            Message 5 of 26 , May 7 1:12 AM
            • 0 Attachment
              On 07-May-2013 05:57 +0200, mattn wrote:

              >> I fully agree, it would be better if :redir could be fixed; even with my
              >> proposed inclusion of split()ing into lines, the additional benefit is
              >> low, and it will take a long time until plugins can fully benefit from
              >> that function without a compatibility wrapper.
              >>
              >> Because I'm still mostly ignorant of the Vim source code, I had assumed
              >> the patch author has found fixing :redir too difficult (as it also /
              >> mostly handles file and register targets where there indeed will be
              >> issues with overriding); I hope mattn can shed some light on that.
              >
              > The functions in message.c write message with color attributes. So if
              > splitting per line, it must get whole contents at the first.

              Well, okay; right now, if split() is called separately, that works out
              the same.

              > I guess someone want to get the contents as text which contains \n,
              > but someone want to get the contents as array of lines. So I want to
              > keep the contents as text.

              My point is that I've *never* seen that use case so far in plugins; they
              all want to parse the captured output into a data structure. Do you have
              any counterexample? (And for that rare use case, they could be join()'ed
              back together easily.)

              Any way, that built-in splitting would only be one weak counterargument
              in favor of a separate capture() function. Could you make :redir work
              with nested invocations instead?!

              -- regards, ingo

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
            • ZyX
              ... Almost never used not splitted messages (they were only needed by me in case there is only one line of output; and only because splitting one line does not
              Message 6 of 26 , May 7 12:44 PM
              • 0 Attachment
                > The functions in message.c write message with color attributes. So if splitting per line, it must get whole contents at the first.
                > I guess someone want to get the contents as text which contains \n, but someone want to get the contents as array of lines. So I want to keep the contents as text.
                >
                > Thanks.

                Almost never used not splitted messages (they were only needed by me in case there is only one line of output; and only because splitting one line does not make much sense; code with proposed by @Ingo Karkat implementation of `capture()` is not a tiny bit harder to write in this case). I have another proposal:

                1. Fix :redir to allow nesting.
                2. Fix :redir to hold reference to dictionary containing variable (like l:, g:, s: or such) to workaround problem I mentioned in https://groups.google.com/forum/#!searchin/vim_dev/redir/vim_dev/1M3QS46SrAk/a13SqNpPWywJ. Obviously only in case redirecting to variable.
                3. Make :redir automatically end when function ends if it holds reference to l:. In any case I doubt anybody needs to start redir in one function and end in another and do redirection to function-local variable at the time. This also must workaround stale :redir's left due to an exception (plugin authors almost always capture to function-local variables).
                4. (optional) Add capture() function, but it should return a list of lists of dictionaries:

                [
                [{'hl': "Error", 'text': "Error on first line"}, {'hl': "OtherHL", 'text': "Other highlight in first line (possible when using :echon)"}],
                [{'hl': "Normal", 'text': "Regular message on second line"}],
                ]

                . It is useful in some cases (like any use of :redir this cases happen mainly because of problems in API: distinguishing between `<LT>Space>` and `<Space>` in :map output (current code I use assumes nobody needs to define mapping like `<LT>F1>`; assumption has not proved false by the time, but having to use it bugs me); distinguishing between regular text and special characters in messages (don’t remember when I needed this, but there was some case)).
                5. (optional) Extend syntax of :redir to capture to a list of lines.

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              • Ingo Karkat
                ... Yes, that s what I thought. Thanks for this data point! ... You bring up very good points; I fully agree. ... I think everyone agrees with this, and it
                Message 7 of 26 , May 7 11:55 PM
                • 0 Attachment
                  On 07-May-2013 21:44 +0200, ZyX wrote:

                  >> The functions in message.c write message with color attributes. So if splitting per line, it must get whole contents at the first.
                  >> I guess someone want to get the contents as text which contains \n, but someone want to get the contents as array of lines. So I want to keep the contents as text.
                  >>
                  >> Thanks.
                  >
                  > Almost never used not splitted messages (they were only needed by me in case there is only one line of output; and only because splitting one line does not make much sense; code with proposed by @Ingo Karkat implementation of `capture()` is not a tiny bit harder to write in this case).

                  Yes, that's what I thought. Thanks for this data point!

                  > I have another proposal:

                  You bring up very good points; I fully agree.

                  > 1. Fix :redir to allow nesting.

                  I think everyone agrees with this, and it should be first priority.

                  > 2. Fix :redir to hold reference to dictionary containing variable (like l:, g:, s: or such) to workaround problem I mentioned in https://groups.google.com/forum/#!searchin/vim_dev/redir/vim_dev/1M3QS46SrAk/a13SqNpPWywJ. Obviously only in case redirecting to variable.

                  I don't fully understand, but as long as it's backwards compatible,
                  additional syntax, why not!?

                  > 3. Make :redir automatically end when function ends if it holds reference to l:. In any case I doubt anybody needs to start redir in one function and end in another and do redirection to function-local variable at the time. This also must workaround stale :redir's left due to an exception (plugin authors almost always capture to function-local variables).

                  This would alleviate a common programming error. Currently, all :redir
                  commands have to be wrapped in try..finally to be safe, but few people
                  do this.

                  > 4. (optional) Add capture() function, but it should return a list of lists of dictionaries:
                  >
                  > [
                  > [{'hl': "Error", 'text': "Error on first line"}, {'hl': "OtherHL", 'text': "Other highlight in first line (possible when using :echon)"}],
                  > [{'hl': "Normal", 'text': "Regular message on second line"}],
                  > ]
                  >
                  > . It is useful in some cases (like any use of :redir this cases happen mainly because of problems in API: distinguishing between `<LT>Space>` and `<Space>` in :map output (current code I use assumes nobody needs to define mapping like `<LT>F1>`; assumption has not proved false by the time, but having to use it bugs me); distinguishing between regular text and special characters in messages (don’t remember when I needed this, but there was some case)).

                  Good idea. I had this need once with the :jumps output: Without the
                  highlighting information, I had to implement a 99%-successful heuristic
                  to decide whether the line refers to the current buffer or another file.
                  It would be even better if the (mostly column-based) information issued
                  by most commands came pre-parsed, but that probably would mean modifying
                  all command implementations.

                  > 5. (optional) Extend syntax of :redir to capture to a list of lines.

                  Since this only applies for redirecting into a variable (not files or
                  registers), I'd rather not further complicate the existing :redir
                  command with this. Instead, if your proposal 4. capture() is
                  implemented, have it take an argument {includeHl}, and just return a
                  list of lines (without the Dictionary stuff) if 0 is passed. Most uses
                  don't need the highlighting; this would make the function applicable to
                  more use cases.


                  I'd love to have all of those good enhancements, but I know that unless
                  someone now starts working on a patch, this great discussion will again
                  fizzle out, like so many before.

                  -- regards, ingo

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php

                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                • ZyX
                  ... It is backwards compatible, but there is no additional syntax. When you call :redir with variable name it seems (I did not look at the code) to create a
                  Message 8 of 26 , May 8 10:57 AM
                  • 0 Attachment
                    > > 1. Fix :redir to allow nesting.
                    >
                    > I think everyone agrees with this, and it should be first priority.
                    >
                    > > 2. Fix :redir to hold reference to dictionary containing variable (like l:, g:, s: or such) to workaround problem I mentioned in https://groups.google.com/forum/#!searchin/vim_dev/redir/vim_dev/1M3QS46SrAk/a13SqNpPWywJ. Obviously only in case redirecting to variable.
                    >
                    > I don't fully understand, but as long as it's backwards compatible,
                    > additional syntax, why not!?

                    It is backwards compatible, but there is no additional syntax. When you call :redir with variable name it seems (I did not look at the code) to create a variable, save variable name (but not the typval_T reference) and when writing to it use usual ways to get variable and write to the structure. Thus if :redir END is called when created variable is not in (script-)local scope of this call it errors out with “variable does not exist” or like. If :redir instead saved reference to the dict_T structure and then used usual methods of assigning dictionary key that error won’t appear. Alternatively this saved reference can be used to check whether :redir END is called in the same scope and do nothing (or give different error message) in case it is not. With nested :redir’ections I have nothing against error, without you will have to wrap `:redir =>` into a separate :try..:catch (with empty :catch block, and with pattern argument in this command) to be safe. In addition to surrounding :try..:finally of course.

                    > > 3. Make :redir automatically end when function ends if it holds reference to l:. In any case I doubt anybody needs to start redir in one function and end in another and do redirection to function-local variable at the time. This also must workaround stale :redir's left due to an exception (plugin authors almost always capture to function-local variables).
                    >
                    > This would alleviate a common programming error. Currently, all :redir
                    > commands have to be wrapped in try..finally to be safe, but few people
                    > do this.
                    >
                    > > 4. (optional) Add capture() function, but it should return a list of lists of dictionaries:
                    > >
                    > > [
                    > > [{'hl': "Error", 'text': "Error on first line"}, {'hl': "OtherHL", 'text': "Other highlight in first line (possible when using :echon)"}],
                    > > [{'hl': "Normal", 'text': "Regular message on second line"}],
                    > > ]
                    > >
                    > > . It is useful in some cases (like any use of :redir this cases happen mainly because of problems in API: distinguishing between `<LT>Space>` and `<Space>` in :map output (current code I use assumes nobody needs to define mapping like `<LT>F1>`; assumption has not proved false by the time, but having to use it bugs me); distinguishing between regular text and special characters in messages (don�t remember when I needed this, but there was some case)).
                    >
                    > Good idea. I had this need once with the :jumps output: Without the
                    > highlighting information, I had to implement a 99%-successful heuristic
                    > to decide whether the line refers to the current buffer or another file.
                    > It would be even better if the (mostly column-based) information issued
                    > by most commands came pre-parsed, but that probably would mean modifying
                    > all command implementations.
                    >
                    > > 5. (optional) Extend syntax of :redir to capture to a list of lines.
                    >
                    > Since this only applies for redirecting into a variable (not files or
                    > registers), I'd rather not further complicate the existing :redir
                    > command with this. Instead, if your proposal 4. capture() is
                    > implemented, have it take an argument {includeHl}, and just return a
                    > list of lines (without the Dictionary stuff) if 0 is passed. Most uses
                    > don't need the highlighting; this would make the function applicable to
                    > more use cases.
                    >
                    > I'd love to have all of those good enhancements, but I know that unless
                    > someone now starts working on a patch, this great discussion will again
                    > fizzle out, like so many before.

                    I am going to reduce the need for :redir for python users: further extension to python interface providing:

                    - vim.maps/vim.buffer.maps (*iterable* and with output similar to maparg(lhs, mode, abbr, dict=True) (but as python object and .noremap with three values in place of two)): removes the need for parsing :redir|:map|:redir END;
                    - vim.commands/vim.buffer.commands (iterable, after finished will be the only way to get command definition using something other then :redir);
                    - vim.autocommands (iterable; same as above);
                    - vim.signs (:sign [un]define/:sign list), vim.buffer.signs (:sign [un]place);
                    - vim.registers (there is no way to get NUL out of the registers except for pasting register in a scratch buffer: any string manipulation transforms internal representation of NUL to "\n" and :redir|:reg truncates output in addition to having to guess (or much more complex: compare with `getreg()` output) whether ^@ meant NUL or somebody was typing ^ then @ like I did here).

                    : this serves two purposes: make plugin writers prefer python (thus I am not talking about VimL API) and create normal API for the things that lack one. If you have other candidates tell me (I guess I already have one from you: :jumps). I will open a RFC here when API is fully designed; though I probably won’t have much time to write an implementation in next one or two months.

                    Maybe I even won’t forget about the discussion after both API and implementation are finished. The thing I won’t add to python though is vim.capture: usage of :redir only means some API is missing and that is the problem that should be fixed.

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/groups/opt_out.
                  • Ingo Karkat
                    ... y..:finally of course. Ah, thanks for the clarification; I think I understand now. I ve never done :redir END in a different scope, but it would be nice if
                    Message 9 of 26 , May 8 11:38 AM
                    • 0 Attachment
                      On 08-May-2013 19:57 +0200, ZyX wrote:

                      >>> 1. Fix :redir to allow nesting.
                      >>
                      >> I think everyone agrees with this, and it should be first priority.
                      >>
                      >>> 2. Fix :redir to hold reference to dictionary containing variable (like l:, g:, s: or such) to workaround problem I mentioned in https://groups.google.com/forum/#!searchin/vim_dev/redir/vim_dev/1M3QS46SrAk/a13SqNpPWywJ. Obviously only in case redirecting to variable.
                      >>
                      >> I don't fully understand, but as long as it's backwards compatible,
                      >> additional syntax, why not!?
                      >
                      > It is backwards compatible, but there is no additional syntax. When you call :redir with variable name it seems (I did not look at the code) to create a variable, save variable name (but not the typval_T reference) and when writing to it use usual ways to get variable and write to the structure. Thus if :redir END is called when created variable is not in (script-)local scope of this call it errors out with “variable does not exist” or like. If :redir instead saved reference to the dict_T structure and then used usual methods of assigning dictionary key that error won’t appear. Alternatively this saved reference can be used to check whether :redir END is called in the same scope and do nothing (or give different error message) in case it is not. With nested :redir’ections I have nothing against error, without you will have to wrap `:redir =>` into a separate :try..:catch (with empty :catch block, and with pattern argument in this command) to be safe. In addition to surrounding :tr
                      y..:finally of course.

                      Ah, thanks for the clarification; I think I understand now. I've never
                      done :redir END in a different scope, but it would be nice if this
                      corner case bug could be fixed.

                      >>> 3. Make :redir automatically end when function ends if it holds reference to l:. In any case I doubt anybody needs to start redir in one function and end in another and do redirection to function-local variable at the time. This also must workaround stale :redir's left due to an exception (plugin authors almost always capture to function-local variables).
                      >>
                      >> This would alleviate a common programming error. Currently, all :redir
                      >> commands have to be wrapped in try..finally to be safe, but few people
                      >> do this.
                      >>
                      >>> 4. (optional) Add capture() function, but it should return a list of lists of dictionaries:
                      >>>
                      >>> [
                      >>> [{'hl': "Error", 'text': "Error on first line"}, {'hl': "OtherHL", 'text': "Other highlight in first line (possible when using :echon)"}],
                      >>> [{'hl': "Normal", 'text': "Regular message on second line"}],
                      >>> ]
                      >>>
                      >>> . It is useful in some cases (like any use of :redir this cases happen mainly because of problems in API: distinguishing between `<LT>Space>` and `<Space>` in :map output (current code I use assumes nobody needs to define mapping like `<LT>F1>`; assumption has not proved false by the time, but having to use it bugs me); distinguishing between regular text and special characters in messages (don�t remember when I needed this, but there was some case)).
                      >>
                      >> Good idea. I had this need once with the :jumps output: Without the
                      >> highlighting information, I had to implement a 99%-successful heuristic
                      >> to decide whether the line refers to the current buffer or another file.
                      >> It would be even better if the (mostly column-based) information issued
                      >> by most commands came pre-parsed, but that probably would mean modifying
                      >> all command implementations.
                      >>
                      >>> 5. (optional) Extend syntax of :redir to capture to a list of lines.
                      >>
                      >> Since this only applies for redirecting into a variable (not files or
                      >> registers), I'd rather not further complicate the existing :redir
                      >> command with this. Instead, if your proposal 4. capture() is
                      >> implemented, have it take an argument {includeHl}, and just return a
                      >> list of lines (without the Dictionary stuff) if 0 is passed. Most uses
                      >> don't need the highlighting; this would make the function applicable to
                      >> more use cases.
                      >>
                      >> I'd love to have all of those good enhancements, but I know that unless
                      >> someone now starts working on a patch, this great discussion will again
                      >> fizzle out, like so many before.
                      >
                      > I am going to reduce the need for :redir for python users: further extension to python interface providing:
                      >
                      > - vim.maps/vim.buffer.maps (*iterable* and with output similar to maparg(lhs, mode, abbr, dict=True) (but as python object and .noremap with three values in place of two)): removes the need for parsing :redir|:map|:redir END;
                      > - vim.commands/vim.buffer.commands (iterable, after finished will be the only way to get command definition using something other then :redir);
                      > - vim.autocommands (iterable; same as above);
                      > - vim.signs (:sign [un]define/:sign list), vim.buffer.signs (:sign [un]place);
                      > - vim.registers (there is no way to get NUL out of the registers except for pasting register in a scratch buffer: any string manipulation transforms internal representation of NUL to "\n" and :redir|:reg truncates output in addition to having to guess (or much more complex: compare with `getreg()` output) whether ^@ meant NUL or somebody was typing ^ then @ like I did here).
                      >
                      > : this serves two purposes: make plugin writers prefer python (thus I am not talking about VimL API) and create normal API for the things that lack one. If you have other candidates tell me (I guess I already have one from you: :jumps). I will open a RFC here when API is fully designed; though I probably won’t have much time to write an implementation in next one or two months.

                      I've seen your many recent Python patches, and it's certainly good to
                      have more things exposed in a native language way. However, I'm a bit
                      concerned if (but I know too little about your patches to be sure) you
                      have to write all the plumbing from Vim's internal structures to the
                      Python interface in a Python-specific way. I would prefer if most of the
                      plumbing were done in Vim core, so that Vimscript as well as other
                      language integrations could easily benefit. I imagine that the object
                      model of Vimscript, Python, Perl, Ruby, Lua, etc. are similar enough
                      that if all that access functionality existed in Vim core, each
                      integration could get at that with a very thin language-dependent bridge.

                      To put it in another way: Christian Brabandt recently implemented a
                      Vimscript getsign() function and extended exists() to be able to access
                      the sign information. I would hope that your vim.signs implementation
                      could leverage most of that from Vim core. It would be bad if your
                      implementation is a completely separate access of the core Vim
                      structures, because of:
                      - duplication = code bloat
                      - other language integrations will eventually want to have the same
                      access; each has to tediously reinvent the wheel, and will probably do
                      so inconsistently
                      - since maintenance is separate for each language, we end up with Python
                      abilities for A, B, and C, Vimscript access for A & B (and use of :redir
                      for C), and e.g. Ruby access only for C.

                      > Maybe I even won’t forget about the discussion after both API and implementation are finished. The thing I won’t add to python though is vim.capture: usage of :redir only means some API is missing and that is the problem that should be fixed.

                      Don't get me wrong; I appreciate your efforts to improve the Python
                      interface. I'm just a bit afraid that in the future, people will use
                      Python just because there's no equally convenient way to access the
                      information from Vimscript. To me, that would be bad. The Vimscript API
                      should remain the first-class, full interface (that doesn't preclude
                      other languages from having an equally rich interface), so that other
                      languages are chosen based on their merits (mostly familiarity and
                      external APIs that not available inside Vim), not based on need.

                      -- regards, ingo

                      --
                      --
                      You received this message from the "vim_dev" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php

                      ---
                      You received this message because you are subscribed to the Google Groups "vim_dev" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                      For more options, visit https://groups.google.com/groups/opt_out.
                    • ZyX
                      ... You can review the patches. Most code is converting arguments and result to/from PyObject* and error-checking, there is not much that can be moved to core.
                      Message 10 of 26 , May 8 1:10 PM
                      • 0 Attachment
                        > I've seen your many recent Python patches, and it's certainly good to
                        > have more things exposed in a native language way. However, I'm a bit
                        > concerned if (but I know too little about your patches to be sure) you
                        > have to write all the plumbing from Vim's internal structures to the
                        > Python interface in a Python-specific way. I would prefer if most of the
                        > plumbing were done in Vim core, so that Vimscript as well as other
                        > language integrations could easily benefit. I imagine that the object
                        > model of Vimscript, Python, Perl, Ruby, Lua, etc. are similar enough
                        > that if all that access functionality existed in Vim core, each
                        > integration could get at that with a very thin language-dependent bridge.

                        You can review the patches. Most code is converting arguments and result to/from PyObject* and error-checking, there is not much that can be moved to core. I can’t generalize things like WindowAttr for e.g. lua: it cannot be used because WindowAttr returns PyObject* and in lua it would “return” void and push to the stack. Not saying I do not know other APIs. Things that make sense to put in core I put in core, but there are not much: in that sequence I only added functions for finding windows, temporary switching to specific window+tabpage and stricter get_option_value.

                        > To put it in another way: Christian Brabandt recently implemented a
                        > Vimscript getsign() function and extended exists() to be able to access
                        > the sign information. I would hope that your vim.signs implementation
                        > could leverage most of that from Vim core. It would be bad if your
                        > implementation is a completely separate access of the core Vim
                        > structures, because of:
                        > - duplication = code bloat

                        As I said I add to core what makes sense to be added there. Most of code in if_py_both is data conversions from C types to PyObject* and back, error handling or other python-specific stuff; I see no candidates for moving to core (obviously does not mean there are no). Code is mostly trivial, but hard to write (for a high-level language user) due to its amount and amount of things that need to be cared about (exceptions, failed memory allocation, decref/incref, rarely alloc/free, etc).

                        > - other language integrations will eventually want to have the same
                        > access; each has to tediously reinvent the wheel, and will probably do
                        > so inconsistently

                        Other languages’ integrations should be consistent with other languages’ best practices, not with python or VimL. Python users would not be happy with vim.lock(dict) in place of setattr(dict, 'locked', True) (dict.locked = True), but VimL uses first variant (with a command though). dict['abc']['def'] may be expected to throw an exception for absent 'abc' or 'def', or return undef and create a dictionary for key 'abc' (and even do this depending on options; I am talking about perl here, though dunno whether you can create a vim.dictionary there).

                        > - since maintenance is separate for each language, we end up with Python
                        > abilities for A, B, and C, Vimscript access for A & B (and use of :redir
                        > for C), and e.g. Ruby access only for C.

                        Yup. It would be good if there was a ruby user+developer to port my changes to ruby though. As I said most code is service one, lua won’t benefit from code that transforms to PyObject* handling errors in the middle, so you can expect extending lua being just as hard (or harder/easier depending on its C API) as extending python, with a small help from a small amount of functions that are general enough to have them in core.

                        > > Maybe I even won�t forget about the discussion after both API and implementation are finished. The thing I won�t add to python though is vim.capture: usage of :redir only means some API is missing and that is the problem that should be fixed.
                        >
                        > Don't get me wrong; I appreciate your efforts to improve the Python
                        > interface. I'm just a bit afraid that in the future, people will use
                        > Python just because there's no equally convenient way to access the
                        > information from Vimscript. To me, that would be bad. The Vimscript API
                        > should remain the first-class, full interface (that doesn't preclude
                        > other languages from having an equally rich interface), so that other
                        > languages are chosen based on their merits (mostly familiarity and
                        > external APIs that not available inside Vim), not based on need.

                        To choose other languages for their familiarity/*external* APIs somebody needs to find another person to write all the things I am implementing for Python and this would be good. I don’t think I will live long enough to see when you could choose language for vim plugin without taking vim<->language API into account. Also don’t forget that A and B that are not in ruby can be accessed through VimL layer.

                        VimL being a first-class is not a good thing because of a big number of issues not fixable due to either backwards compatibility or too big codebase changes that require to fix them; if I was sure vim users are required to have some language but VimL I would immediately switch to it. Though you have a point about another languages: things have to be added to VimL because this way they automatically “appear” in other supported languages.

                        Note though that VimL additions I can design will much likely smell python, but python has iterators to walk through vim.mappings without allocating a big list by a small chunks (you can’t do one alloc for all list items in C code and expect it not to crash as a consequence) and tp_as_mapping->mp_subscript to not allocate a big nested dictionary just for querying for autocommands['Powerline']['VimEnter']['*']. When you will see my VimL APIs in RFC please be captious.

                        --
                        --
                        You received this message from the "vim_dev" maillist.
                        Do not top-post! Type your reply below the text you are replying to.
                        For more information, visit http://www.vim.org/maillist.php

                        ---
                        You received this message because you are subscribed to the Google Groups "vim_dev" group.
                        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                        For more options, visit https://groups.google.com/groups/opt_out.
                      • Ingo Karkat
                        ... Good, I m relieved. ... Yes, due to the mismatch between the C code in core and the mostly object-oriented APIs of other languages, I had assumed that
                        Message 11 of 26 , May 8 2:29 PM
                        • 0 Attachment
                          On 08-May-2013 22:10 +0200, ZyX wrote:

                          >> I've seen your many recent Python patches, and it's certainly good to
                          >> have more things exposed in a native language way. However, I'm a bit
                          >> concerned if (but I know too little about your patches to be sure) you
                          >> have to write all the plumbing from Vim's internal structures to the
                          >> Python interface in a Python-specific way. I would prefer if most of the
                          >> plumbing were done in Vim core, so that Vimscript as well as other
                          >> language integrations could easily benefit. I imagine that the object
                          >> model of Vimscript, Python, Perl, Ruby, Lua, etc. are similar enough
                          >> that if all that access functionality existed in Vim core, each
                          >> integration could get at that with a very thin language-dependent bridge.
                          >
                          > You can review the patches. Most code is converting arguments and result to/from PyObject* and error-checking, there is not much that can be moved to core. I can’t generalize things like WindowAttr for e.g. lua: it cannot be used because WindowAttr returns PyObject* and in lua it would “return” void and push to the stack. Not saying I do not know other APIs. Things that make sense to put in core I put in core, but there are not much: in that sequence I only added functions for finding windows, temporary switching to specific window+tabpage and stricter get_option_value.

                          Good, I'm relieved.

                          >> To put it in another way: Christian Brabandt recently implemented a
                          >> Vimscript getsign() function and extended exists() to be able to access
                          >> the sign information. I would hope that your vim.signs implementation
                          >> could leverage most of that from Vim core. It would be bad if your
                          >> implementation is a completely separate access of the core Vim
                          >> structures, because of:
                          >> - duplication = code bloat
                          >
                          > As I said I add to core what makes sense to be added there. Most of code in if_py_both is data conversions from C types to PyObject* and back, error handling or other python-specific stuff; I see no candidates for moving to core (obviously does not mean there are no). Code is mostly trivial, but hard to write (for a high-level language user) due to its amount and amount of things that need to be cared about (exceptions, failed memory allocation, decref/incref, rarely alloc/free, etc).

                          Yes, due to the mismatch between the C code in core and the mostly
                          object-oriented APIs of other languages, I had assumed that there is
                          quite some boilerplate code. If any of that could be wrapped for reuse
                          [by other language integrations], either through macros or helper
                          functions, that would be desirable.

                          >> - other language integrations will eventually want to have the same
                          >> access; each has to tediously reinvent the wheel, and will probably do
                          >> so inconsistently
                          >
                          > Other languages’ integrations should be consistent with other languages’ best practices, not with python or VimL. Python users would not be happy with vim.lock(dict) in place of setattr(dict, 'locked', True) (dict.locked = True), but VimL uses first variant (with a command though). dict['abc']['def'] may be expected to throw an exception for absent 'abc' or 'def', or return undef and create a dictionary for key 'abc' (and even do this depending on options; I am talking about perl here, though dunno whether you can create a vim.dictionary there).
                          >
                          >> - since maintenance is separate for each language, we end up with Python
                          >> abilities for A, B, and C, Vimscript access for A & B (and use of :redir
                          >> for C), and e.g. Ruby access only for C.
                          >
                          > Yup. It would be good if there was a ruby user+developer to port my changes to ruby though. As I said most code is service one, lua won’t benefit from code that transforms to PyObject* handling errors in the middle, so you can expect extending lua being just as hard (or harder/easier depending on its C API) as extending python, with a small help from a small amount of functions that are general enough to have them in core.
                          >
                          >>> Maybe I even won�t forget about the discussion after both API and implementation are finished. The thing I won�t add to python though is vim.capture: usage of :redir only means some API is missing and that is the problem that should be fixed.
                          >>
                          >> Don't get me wrong; I appreciate your efforts to improve the Python
                          >> interface. I'm just a bit afraid that in the future, people will use
                          >> Python just because there's no equally convenient way to access the
                          >> information from Vimscript. To me, that would be bad. The Vimscript API
                          >> should remain the first-class, full interface (that doesn't preclude
                          >> other languages from having an equally rich interface), so that other
                          >> languages are chosen based on their merits (mostly familiarity and
                          >> external APIs that not available inside Vim), not based on need.
                          >
                          > To choose other languages for their familiarity/*external* APIs somebody needs to find another person to write all the things I am implementing for Python and this would be good. I don’t think I will live long enough to see when you could choose language for vim plugin without taking vim<->language API into account. Also don’t forget that A and B that are not in ruby can be accessed through VimL layer.
                          >
                          > VimL being a first-class is not a good thing because of a big number of issues not fixable due to either backwards compatibility or too big codebase changes that require to fix them; if I was sure vim users are required to have some language but VimL I would immediately switch to it. Though you have a point about another languages: things have to be added to VimL because this way they automatically “appear” in other supported languages.

                          I've heard the dislike of Vimscript, but much of that is probably due to
                          the developers' relative unfamiliarity with it. For extensions that are
                          closely related to text-editing, I found it quite nice and powerful, and
                          there is no mismatch between the Vim API and another language to bridge.
                          But I agree that other stuff (e.g. any interfacing with external systems
                          or networking stuff) is extremely hard (though people have written
                          hashing functions or JSON parsers in Vimscript).

                          > Note though that VimL additions I can design will much likely smell python, but python has iterators to walk through vim.mappings without allocating a big list by a small chunks (you can’t do one alloc for all list items in C code and expect it not to crash as a consequence) and tp_as_mapping->mp_subscript to not allocate a big nested dictionary just for querying for autocommands['Powerline']['VimEnter']['*']. When you will see my VimL APIs in RFC please be captious.

                          Certainly, Vimscript won't win any performance contest; other language
                          interpreters are probably way more optimized.


                          Thanks for all your explanations; you indeed have very deep knowledge in
                          this area. I'm looking forward to your patches and improved Python
                          integration! Good luck!

                          -- regards, ingo

                          --
                          --
                          You received this message from the "vim_dev" maillist.
                          Do not top-post! Type your reply below the text you are replying to.
                          For more information, visit http://www.vim.org/maillist.php

                          ---
                          You received this message because you are subscribed to the Google Groups "vim_dev" group.
                          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                          For more options, visit https://groups.google.com/groups/opt_out.
                        • ZyX
                          ... To do this I need to know other languages APIs. And have to create a “header” file if_interp.h really containing C code like if_py_both.h (never added
                          Message 12 of 26 , May 8 3:54 PM
                          • 0 Attachment
                            > > As I said I add to core what makes sense to be added there. Most of code in if_py_both is data conversions from C types to PyObject* and back, error handling or other python-specific stuff; I see no candidates for moving to core (obviously does not mean there are no). Code is mostly trivial, but hard to write (for a high-level language user) due to its amount and amount of things that need to be cared about (exceptions, failed memory allocation, decref/incref, rarely alloc/free, etc).
                            >
                            > Yes, due to the mismatch between the C code in core and the mostly
                            > object-oriented APIs of other languages, I had assumed that there is
                            > quite some boilerplate code. If any of that could be wrapped for reuse
                            > [by other language integrations], either through macros or helper
                            > functions, that would be desirable.

                            To do this I need to know other languages APIs. And have to create a “header” file if_interp.h really containing C code like if_py_both.h (never added any files to vim except for tests and runtime ones). And have to write something that uses such functions in e.g. if_ruby.c: there is no other way I can prove this generalization necessary. Thus if some ruby developer will want to extend ruby in a similar way it would be he who should write generalization: I just do not have enough knowledge and time to do this.

                            > I've heard the dislike of Vimscript, but much of that is probably due to
                            > the developers' relative unfamiliarity with it. For extensions that are
                            > closely related to text-editing, I found it quite nice and powerful, and
                            > there is no mismatch between the Vim API and another language to bridge.
                            > But I agree that other stuff (e.g. any interfacing with external systems
                            > or networking stuff) is extremely hard (though people have written
                            > hashing functions or JSON parsers in Vimscript).

                            This is not a problem with other stuff. I have to care about a big bunch of options like &ignorecase. I have to always remember about strange defaults (:map without nore while best practices tell nore should be default, :command without -bar while most commands will be more convenient to use with it and so on). I have to remember about escaping, and correct escaping (three common escaping variants for different cases). I have to forget about writing some neat code because it does not work reliably (e.g. :!python %). I have to generate a big number of lines just to take care of NUL case when in python it is just a regular symbol. Even in zsh it looks like a regular symbol though zsh uses just the same NUL-terminated strings. I have to know about some inconsistencies which will never be fixed due to backwards compatibility. Do you know that there is no option for filename completion in user-defined commands except for `custom…`? I have to hope user have sane configuration. Did you ever hear python `subprocess` module refusing to work because user has wrong configuration options? Also try to write down rules for dividing a string into a list of statements for python and for VimL and compare it. There is no such thing like a bunch of variants of handling newline depending on the bunch of variants of non-immediately-surrounding context† in python.

                            By the way, I have written both (adler32 hash (math was easier then base64 encoder/decoder which I used to write too) and both YAML (superset of JSON) and JSON parsers). And even a DSL for describing (and validating and completing) function and command arguments.

                            Developers do not like VimL not because they do not know it. They dislike it because they do. I cannot say there are no things that bug me in python, zsh or perl (high-level languages I know good enough to judge, best known first), but VimL is the only one that bugs me that much. I have to say though that when I was writing some simple things it seemed good, but “the farther into the forest, the thicker the trees”. I just did not know all the cases where my simple things can break and did not care about them.

                            † (NL in file, NL in file followed by spaces and backslash, NL in execute after commands not taking | as an argument, NL in execute after other commands, NL in execute after (built-in!) command taking expression (:echo) and unclosed quote, NL in execute after :function): main questions to answer are whether we are in a file, in a Ex prompt (BTW, have not explored this), in an :execute or in a function inside :execute. And what is the command we are computing arguments for.

                            > > Note though that VimL additions I can design will much likely smell python, but python has iterators to walk through vim.mappings without allocating a big list by a small chunks (you can�t do one alloc for all list items in C code and expect it not to crash as a consequence) and tp_as_mapping->mp_subscript to not allocate a big nested dictionary just for querying for autocommands['Powerline']['VimEnter']['*']. When you will see my VimL APIs in RFC please be captious.
                            >
                            > Certainly, Vimscript won't win any performance contest; other language
                            > interpreters are probably way more optimized.

                            They are. Problem is not with optimizations, problem is some set of features like concurrent processing and a number of places where globals are used.

                            > Thanks for all your explanations; you indeed have very deep knowledge in
                            > this area. I'm looking forward to your patches and improved Python
                            > integration! Good luck!

                            --
                            --
                            You received this message from the "vim_dev" maillist.
                            Do not top-post! Type your reply below the text you are replying to.
                            For more information, visit http://www.vim.org/maillist.php

                            ---
                            You received this message because you are subscribed to the Google Groups "vim_dev" group.
                            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                            For more options, visit https://groups.google.com/groups/opt_out.
                          • mattn
                            On Thursday, May 9, 2013 7:54:38 AM UTC+9, ZyX wrote: I don t hope that capture() contains highlight attributes. It should be plain text. And I guess
                            Message 13 of 26 , May 9 7:09 PM
                            • 0 Attachment
                              On Thursday, May 9, 2013 7:54:38 AM UTC+9, ZyX wrote:
                              <snip>

                              I don't hope that capture() contains highlight attributes. It should be plain text. And I guess fixing redir nesting has problems.

                              1. There are plugins that depend on original behavior. This break compatible.
                              2. If skipped 'redir END' with exceptions, we can't close the nesting.

                              For about 2, one of solution is adding bang like 'redir! END'. but this will close all of redir nesting. So it's become un-expected behavior.

                              Thanks.
                              - Yasuhiro Matsumoto

                              --
                              --
                              You received this message from the "vim_dev" maillist.
                              Do not top-post! Type your reply below the text you are replying to.
                              For more information, visit http://www.vim.org/maillist.php

                              ---
                              You received this message because you are subscribed to the Google Groups "vim_dev" group.
                              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                              For more options, visit https://groups.google.com/groups/opt_out.
                            • ZyX
                              ... Show me them then. It breaks only with nested redirections and it breaks only in the fashion that you get more expected output in place of error and empty
                              Message 14 of 26 , May 10 4:18 AM
                              • 0 Attachment
                                пятница, 10 мая 2013 г., 6:09:15 UTC+4 пользователь mattn написал:
                                > On Thursday, May 9, 2013 7:54:38 AM UTC+9, ZyX wrote:
                                > <snip>
                                >
                                > I don't hope that capture() contains highlight attributes. It should be plain text. And I guess fixing redir nesting has problems.
                                >
                                > 1. There are plugins that depend on original behavior. This break compatible.

                                Show me them then. It breaks only with nested redirections and it breaks only in the fashion that you get more expected output in place of error and empty string in variable. There is only one case when nested :redir breaks in other fashion: a sequence of :redir's in one function without :redir ENDs except for last one. I have never seen such a strange thing, but there is a way to fix this case: make :redir update variable immediately each time new :redir is called without waiting for :redir END (in any case you need something to store text redirected at this level, why not make it store in variable/register/file? I do not think anybody benefits from "The variable will remain empty until redirection ends."). Unneeded :redirs will likely be GCed when function ends (third suggestion).

                                > 2. If skipped 'redir END' with exceptions, we can't close the nesting.

                                This is why I have 3-rd suggestion. Most of the time :redir is used to redirect to variables.

                                >
                                > For about 2, one of solution is adding bang like 'redir! END'. but this will close all of redir nesting. So it's become un-expected behavior.

                                You don't need :redir! END. Just don't leave all redirection options to user besides for redirection to variable inside a function. Or use :try..:finally.

                                If somebody has left stale redirections currently you will see unexpected behavior (error) with regular :redir, refer to the link in my first message. In addition to the text logged somewhere (but only to one place) until next :redir is called.

                                If somebody has left stale redirections with new proposal you will just have text logged somewhere (but only to one place as well) until session ends. It is very rare to see stale redirections: I have never seen them so far and never written something that leaves such redirections; and logging to some place with usual amount of messages is not too harmful. Thus advantages are greater then disadvantages.

                                To diagnose the issue :redir (without arguments) should print the stack of places where redirected text will go in order; it will only be actually written to the one last position.

                                >
                                > Thanks.
                                > - Yasuhiro Matsumoto

                                --
                                --
                                You received this message from the "vim_dev" maillist.
                                Do not top-post! Type your reply below the text you are replying to.
                                For more information, visit http://www.vim.org/maillist.php

                                ---
                                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                For more options, visit https://groups.google.com/groups/opt_out.
                              • Ingo Karkat
                                On 10-May-2013 04:09 +0200, mattn wrote: I beg your pardon, but I find your entire argument weak and unconstructive. ... Both ZyX s and my recent messages have
                                Message 15 of 26 , May 13 1:02 PM
                                • 0 Attachment
                                  On 10-May-2013 04:09 +0200, mattn wrote:

                                  I beg your pardon, but I find your entire argument weak and unconstructive.

                                  > On Thursday, May 9, 2013 7:54:38 AM UTC+9, ZyX wrote:
                                  > <snip>
                                  >
                                  > I don't hope that capture() contains highlight attributes. It should
                                  > be plain text.

                                  Both ZyX's and my recent messages have already shown cases where this
                                  additional information is essential. If you don't need it, ignore the
                                  information / don't request it (my suggestion makes this optional via a
                                  passed flag).

                                  > And I guess fixing redir nesting has problems.

                                  Your intuitions and guesses aren't really helpful in this otherwise very
                                  factual discussion.

                                  > 1. There are plugins that depend on original behavior. This break compatible.

                                  I cannot think of a plugin that depends on having a nested :redir throw
                                  an error. That's just bad coding practice, and it could easily be
                                  corrected in the plugin. Do you have an actual example? As I've shown,
                                  contrarily there are plugins that (when working together) are indeed
                                  broken due to what I would call an implementation deficiency.

                                  > [3 lines deleted]

                                  Please take no offense, I value your opinion like every other, but
                                  please support them with facts and concrete examples as much as
                                  possible. Thank you.

                                  -- regards, ingo

                                  --
                                  --
                                  You received this message from the "vim_dev" maillist.
                                  Do not top-post! Type your reply below the text you are replying to.
                                  For more information, visit http://www.vim.org/maillist.php

                                  ---
                                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                  For more options, visit https://groups.google.com/groups/opt_out.
                                • Ingo Karkat
                                  ... ever hear python `subprocess` module refusing to work because user has wrong configuration options? Also try to write down rules for dividing a string into
                                  Message 16 of 26 , May 13 1:43 PM
                                  • 0 Attachment
                                    On 09-May-2013 00:54 +0200, ZyX wrote:

                                    > > > [10 sentences deleted]
                                    >
                                    >> I've heard the dislike of Vimscript, but much of that is probably due to
                                    >> the developers' relative unfamiliarity with it. For extensions that are
                                    >> closely related to text-editing, I found it quite nice and powerful, and
                                    >> there is no mismatch between the Vim API and another language to bridge.
                                    >> But I agree that other stuff (e.g. any interfacing with external systems
                                    >> or networking stuff) is extremely hard (though people have written
                                    >> hashing functions or JSON parsers in Vimscript).
                                    >
                                    > This is not a problem with other stuff. I have to care about a big bunch of options like &ignorecase. I have to always remember about strange defaults (:map without nore while best practices tell nore should be default, :command without -bar while most commands will be more convenient to use with it and so on). I have to remember about escaping, and correct escaping (three common escaping variants for different cases). I have to forget about writing some neat code because it does not work reliably (e.g. :!python %). I have to generate a big number of lines just to take care of NUL case when in python it is just a regular symbol. Even in zsh it looks like a regular symbol though zsh uses just the same NUL-terminated strings. I have to know about some inconsistencies which will never be fixed due to backwards compatibility. Do you know that there is no option for filename completion in user-defined commands except for `custom…`? I have to hope user have sane configuration. Did you
                                    ever hear python `subprocess` module refusing to work because user has wrong configuration options? Also try to write down rules for dividing a string into a list of statements for python and for VimL and compare it. There is no such thing like a bunch of variants of handling newline depending on the bunch of variants of non-immediately-surrounding context† in python.

                                    Good explanation of the pain points. My take on this is: Vimscript is
                                    made primarily for interactive use, and therefore it is difficult to
                                    write watertight plugin code that works under all circumstances.

                                    It's great to be able to write :! python % when you know the Python in
                                    your PATH is the right one, and your file doesn't contain spaces, but in
                                    a plugin, you have to pull together configuration variables,
                                    executable(), fnamescape(), etc., and it gets messy quickly. Likewise,
                                    :map is often right for ad-hoc mappings, but all plugins should use
                                    :noremap. Same for 'ignorecase'.

                                    > [2 sentences deleted]
                                    >
                                    > Developers do not like VimL not because they do not know it. They dislike it because they do. I cannot say there are no things that bug me in python, zsh or perl (high-level languages I know good enough to judge, best known first), but VimL is the only one that bugs me that much. I have to say though that when I was writing some simple things it seemed good, but “the farther into the forest, the thicker the trees”. I just did not know all the cases where my simple things can break and did not care about them.

                                    For me, it's again the trade off (or historical accident) that was made
                                    in favor of easy migration from ad-hoc macros to mappings to
                                    full-fledged personal plugins to published plugins. The great thing is
                                    that it's Vimscript all down that path, the downside is that what was a
                                    simple one-liner has to be rewritten to a complex dozen-line function
                                    (with stuff that has to be tediously learned) in order to be fully
                                    robust under most users' customizations.

                                    On the other hand, maybe we're holding ourselves to too high ideals.
                                    Most plugins I've tried and reviewed (even from venerable long-time
                                    contributors like Tim Pope or Dr. Chip, though admittedly much less so)
                                    have certain deficiencies or break when used with obscure settings or
                                    environments, but most people are happy to use them, anyway. So I tend
                                    to defend Vimscript on its "worse is better" philosophy when language
                                    purists try to bash it too much. But I would be happy if a different
                                    approach (like your planned much-extended Python interface) turns out to
                                    be as productive without so many pitfalls.

                                    > [11 sentences deleted]

                                    -- regards, ingo

                                    --
                                    --
                                    You received this message from the "vim_dev" maillist.
                                    Do not top-post! Type your reply below the text you are replying to.
                                    For more information, visit http://www.vim.org/maillist.php

                                    ---
                                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                    For more options, visit https://groups.google.com/groups/opt_out.
                                  • ZyX
                                    ... Refer to my message. Nested :redir throws an error only when context is switched, it is possible to have redir = commands command redir = mappings map
                                    Message 17 of 26 , May 13 1:46 PM
                                    • 0 Attachment
                                      > I cannot think of a plugin that depends on having a nested :redir throw
                                      > an error.

                                      Refer to my message. Nested :redir throws an error only when context is switched, it is possible to have

                                      redir => commands
                                      command
                                      redir => mappings
                                      map
                                      redir => imappings
                                      imap
                                      " and so on
                                      redir END

                                      > That's just bad coding practice, and it could easily be
                                      > corrected in the plugin.

                                      I completely agree there. Also refer to my message, I explained there how this situation may easily be worked around (just making it always write to target when any :redir is issued is enough for plugins to work; finishing redirections to local variables after function finishes will delete stale redirs in all cases that make sense to write from my point of view and even not finishing should be rare enough and not result in much harm: at maximum vim killed by OOM killer or a few MiBs long file). I have never actually seen plugins with the above code though.

                                      > Do you have an actual example? As I've shown,
                                      > contrarily there are plugins that (when working together) are indeed
                                      > broken due to what I would call an implementation deficiency.

                                      --
                                      --
                                      You received this message from the "vim_dev" maillist.
                                      Do not top-post! Type your reply below the text you are replying to.
                                      For more information, visit http://www.vim.org/maillist.php

                                      ---
                                      You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                      For more options, visit https://groups.google.com/groups/opt_out.
                                    • mattn
                                      ... I m thinking capture() is not replacement of redir. So I m thinking this issue should be separated with redir s issue. 1. The redir have a problem
                                      Message 18 of 26 , Jun 16, 2013
                                      • 0 Attachment
                                        On Tuesday, May 14, 2013 5:46:25 AM UTC+9, ZyX wrote:
                                        > > I cannot think of a plugin that depends on having a nested :redir throw
                                        > > an error.
                                        >
                                        > Refer to my message. Nested :redir throws an error only when context is switched, it is possible to have
                                        >
                                        > redir => commands
                                        > command
                                        > redir => mappings
                                        > map
                                        > redir => imappings
                                        > imap
                                        > " and so on
                                        > redir END
                                        >
                                        > > That's just bad coding practice, and it could easily be
                                        > > corrected in the plugin.
                                        >
                                        > I completely agree there. Also refer to my message, I explained there how this situation may easily be worked around (just making it always write to target when any :redir is issued is enough for plugins to work; finishing redirections to local variables after function finishes will delete stale redirs in all cases that make sense to write from my point of view and even not finishing should be rare enough and not result in much harm: at maximum vim killed by OOM killer or a few MiBs long file). I have never actually seen plugins with the above code though.
                                        >
                                        > > Do you have an actual example? As I've shown,
                                        > > contrarily there are plugins that (when working together) are indeed
                                        > > broken due to what I would call an implementation deficiency.


                                        I'm thinking capture() is not replacement of redir. So I'm thinking this issue should be separated with redir's issue.

                                        1. The redir have a problem currently, But capture() doesn't.
                                        2. Reasonable to implement.
                                        3. Easy to use. Possible to write per one line.
                                        4. I know there are some users who want capture().

                                        I'm thinking it's good idea to solve some problem.

                                        Thanks.

                                        --
                                        --
                                        You received this message from the "vim_dev" maillist.
                                        Do not top-post! Type your reply below the text you are replying to.
                                        For more information, visit http://www.vim.org/maillist.php

                                        ---
                                        You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                        For more options, visit https://groups.google.com/groups/opt_out.
                                      • Shougo
                                        ... Yes. I think so, too. 1. I can use capture() easily than :redir. Example: let foo = capture( :mes ) redir = foo mes redir END Which is the easy to
                                        Message 19 of 26 , Jun 16, 2013
                                        • 0 Attachment
                                          2013年6月17日月曜日 11時49分10秒 UTC+9 mattn:
                                          > On Tuesday, May 14, 2013 5:46:25 AM UTC+9, ZyX wrote:
                                          > > > I cannot think of a plugin that depends on having a nested :redir throw
                                          > > > an error.
                                          > >
                                          > > Refer to my message. Nested :redir throws an error only when context is switched, it is possible to have
                                          > >
                                          > > redir => commands
                                          > > command
                                          > > redir => mappings
                                          > > map
                                          > > redir => imappings
                                          > > imap
                                          > > " and so on
                                          > > redir END
                                          > >
                                          > > > That's just bad coding practice, and it could easily be
                                          > > > corrected in the plugin.
                                          > >
                                          > > I completely agree there. Also refer to my message, I explained there how this situation may easily be worked around (just making it always write to target when any :redir is issued is enough for plugins to work; finishing redirections to local variables after function finishes will delete stale redirs in all cases that make sense to write from my point of view and even not finishing should be rare enough and not result in much harm: at maximum vim killed by OOM killer or a few MiBs long file). I have never actually seen plugins with the above code though.
                                          > >
                                          > > > Do you have an actual example? As I've shown,
                                          > > > contrarily there are plugins that (when working together) are indeed
                                          > > > broken due to what I would call an implementation deficiency.
                                          >
                                          >
                                          > I'm thinking capture() is not replacement of redir. So I'm thinking this issue should be separated with redir's issue.
                                          >
                                          > 1. The redir have a problem currently, But capture() doesn't.
                                          > 2. Reasonable to implement.
                                          > 3. Easy to use. Possible to write per one line.
                                          > 4. I know there are some users who want capture().
                                          >
                                          > I'm thinking it's good idea to solve some problem.
                                          >
                                          > Thanks.

                                          Yes. I think so, too.

                                          1. I can use capture() easily than :redir.

                                          Example:
                                          let foo = capture(":mes")

                                          redir => foo
                                          mes
                                          redir END

                                          Which is the easy to understand?

                                          2. And, the capture() is used for function arguments.

                                          Example:
                                          echo remote_expr("GVIM1", "capture(':mes')")

                                          I think function is better than command to use it in Vim script.

                                          3. I think the patch works well(because I tested it).
                                          But to fix redir is no patches, it is too far to use it in my plugins.

                                          --
                                          --
                                          You received this message from the "vim_dev" maillist.
                                          Do not top-post! Type your reply below the text you are replying to.
                                          For more information, visit http://www.vim.org/maillist.php

                                          ---
                                          You received this message because you are subscribed to the Google Groups "vim_dev" group.
                                          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                                          For more options, visit https://groups.google.com/groups/opt_out.
                                        Your message has been successfully submitted and would be delivered to recipients shortly.