Loading ...
Sorry, an error occurred while loading the content.
 

Interrupting a python script

Expand Messages
  • Ernie Rael
    The discussion Asynchronous functions (settimeout, setinterval, and cancelinterval) reminded me of some past discussion on interrupting a looping python. I
    Message 1 of 10 , Sep 3, 2013
      The discussion "Asynchronous functions (settimeout, setinterval, and
      cancelinterval)" reminded me of some past discussion on interrupting a
      looping python. I don't remember the details; my recollection is that
      vim checks the input character (rather than catching the signal) and if
      it is the interrupt character then the running vim operation is stopped.
      If there is a looping python, then vim's main loop can not process the
      input character queue and there is no way to notify python to stop.
      There may also be a vim versus gvim issue.

      It seems that if the SIGINT is always caught/handled and only used to
      set a flag such as: interrupt_pending = TRUE, then vim's main loop could
      do something like

      if(interrupt_pending == TRUE) {
      interrupt_pending = FALSE;
      /* do vim's process interrupt stuff */
      }

      which should preserve existing behavior by doing the interrupt handling
      in the main loop.

      Then for handling the python situation, vim's interrupt processing would
      have to invoke something in the python interface that in turn invokes
      PyErr_SetInterrupt() if python is running.

      I'm not familiar with any of this code (vim or python interface) but I
      wanted to put this out so I could drop it from my mental list.

      -ernie

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • Ernie Rael
      ... Doh! Vim s main loop doesn t run, so vim s interrupt processing would never run; the previous sentence is nonsense. For python support for handling
      Message 2 of 10 , Sep 3, 2013
        On 9/3/2013 3:09 PM, Ernie Rael wrote:
        > The discussion "Asynchronous functions (settimeout, setinterval, and
        > cancelinterval)" reminded me of some past discussion on interrupting a
        > looping python. I don't remember the details; my recollection is that
        > vim checks the input character (rather than catching the signal) and
        > if it is the interrupt character then the running vim operation is
        > stopped. If there is a looping python, then vim's main loop can not
        > process the input character queue and there is no way to notify python
        > to stop. There may also be a vim versus gvim issue.
        >
        > It seems that if the SIGINT is always caught/handled and only used to
        > set a flag such as: interrupt_pending = TRUE, then vim's main loop
        > could do something like
        >
        > if(interrupt_pending == TRUE) {
        > interrupt_pending = FALSE;
        > /* do vim's process interrupt stuff */
        > }
        >
        > which should preserve existing behavior by doing the interrupt
        > handling in the main loop.
        >
        > Then for handling the python situation, vim's interrupt processing
        > would have to invoke something in the python interface that in turn
        > invokes PyErr_SetInterrupt() if python is running.
        Doh! Vim's main loop doesn't run, so vim's interrupt processing would
        never run; the previous sentence is nonsense. For python support for
        handling interrupt, the signal handler would have to invoke the python
        interface and that would only work if PyErr_SetInterrupt() does no more
        than sets a flag that is noticed by the python interpreters main loop.
        >
        > I'm not familiar with any of this code (vim or python interface) but I
        > wanted to put this out so I could drop it from my mental list.
        >
        > -ernie
        >

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
      • James McCoy
        ... Something like this would also address the problem where doing something like :call taglist( a ) with a large tags file (e.g., all of /usr/include) takes
        Message 3 of 10 , Sep 3, 2013
          On Tue, Sep 03, 2013 at 03:09:11PM -0700, Ernie Rael wrote:
          > It seems that if the SIGINT is always caught/handled and only used
          > to set a flag such as: interrupt_pending = TRUE, then vim's main
          > loop could do something like
          >
          > if(interrupt_pending == TRUE) {
          > interrupt_pending = FALSE;
          > /* do vim's process interrupt stuff */
          > }

          Something like this would also address the problem where doing something
          like ":call taglist('a')" with a large tags file (e.g., all of
          /usr/include) takes multiple minutes to return. This happens because
          Vim currently uses select() to monitor for the <C-c> keypress while the
          tag processing is happening and those calls to select() dominate the
          run time.

          Cheers,
          --
          James
          GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy <jamessan@...>
        • Marc Weber
          This is my message you ve been looking for: https://groups.google.com/forum/#!searchin/vim_dev/Ruby$20abort$20vim$20marc/vim_dev/irITPpKnTP8/Osl0AHJUH60J
          Message 4 of 10 , Sep 3, 2013
            This is my message you've been looking for:
            https://groups.google.com/forum/#!searchin/vim_dev/Ruby$20abort$20vim$20marc/vim_dev/irITPpKnTP8/Osl0AHJUH60J

            You're looking for PySetInterrupt();

            For ruby there is an equivalent.

            Fixing it would require diving into the signal handling, and I didn't
            understand it - either it has to be refactored and defined well when
            what should happen or ...

            If you'd like to collaborate on fixing this let me know.

            For me alone it'll be too much work. Who else would love to join and
            help fixing this?

            The main problem is that everything is based on a huge hack:
            gui_process_events which gets called even when viml is running, so that
            ctrl-c can be "looked ahead" and viml can be aborted.
            This gui_process_events like functions has side effects which I think it
            should not have, see thread linked above about resize which is not used
            often enough to really care about.

            Which is the corret way to fix this? Have a separate gtk thread which
            gets keyboard input, processes it, and when finding a ctrl-c sets
            PySetInterrupt causing python to stop (running in a different thread).

            And that's where/why I stopped - I had the feeling that fixing this
            would require adding multi threading support or such - thus rewriting
            much gui code. I personally think its the future - but I cannot do it
            alone. Does anybody have different views / ideas / solutions ?

            Marc Weber

            --
            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php

            ---
            You received this message because you are subscribed to the Google Groups "vim_dev" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
          • Ernie Rael
            ... My comments have been ill-informed. I just took a brief look at the vim code and read the threads you point out; thanks. The following comments are more in
            Message 5 of 10 , Sep 5, 2013
              On 9/3/2013 6:34 PM, Marc Weber wrote:
              > This is my message you've been looking for:
              > https://groups.google.com/forum/#!searchin/vim_dev/Ruby$20abort$20vim$20marc/vim_dev/irITPpKnTP8/Osl0AHJUH60J
              >
              > You're looking for PySetInterrupt();
              >
              > For ruby there is an equivalent.
              >
              > Fixing it would require diving into the signal handling, and I didn't
              > understand it - either it has to be refactored and defined well when
              > what should happen or ...

              My comments have been ill-informed. I just took a brief look at the vim
              code and read the threads you point out; thanks. The following comments
              are more in line with the reality of the vim code (but may still be off
              base). I'm assuming that vim/python runs in the same thread; when python
              executes, vim does not executing until the python call returns.

              Of course vim must "normally" run in RAW mode so that it sees all the
              keyboard action, and so can not usually get interrupts. Vim turns on
              interrupts when it executes an external command. I think the following
              approach requires minimal changes to vim's structure and still handles
              the issue.

              - the python interface enables interrupts when it executes python,
              something like mch_call_shell with option SHELL_COOKED.
              - the SIGINT handler informs python to stop

              The SIGINT handler is already minimal (considering only os_unix.c), it
              could be changed to something like:

              signal(SIGINT, (RETSIGTYPE (*)())catch_sigint);
              #if PYTHON_SUPPORT
              if(pthon_busy)
              do_python_interrupt_stuff();
              #endif
              got_int = TRUE;
              SIGRETURN;

              The above is only for discussion (not for compilation). This always
              sets got_int even if python_busy, should be OK and might be safest. The
              check of python_busy in this function is for clarity; efficiency is not
              an issue in this function so could simply call
              python_handle_interrupt_signal() that check state if/as needed (it may
              be that PySetInterrupt. can be invoked even if the python interpreter is
              not running).

              There may be subtleties in enabling interrupts before calling python,
              but it might be as simple as

              int tmode = cur_tmode;
              settmode(TMODE_COOK);
              execute_python_stuff_as_usual_and_set_the_python_busy_flag();
              settmode(tmode); /* may be this is always TMODE_RAW */


              >
              > If you'd like to collaborate on fixing this let me know.

              I've never contributed to vim, and only occasionally use python (never
              with vim). I'm not a good candidate. But I really enjoy considering real
              time issues.

              Note that this does not consider the broader questions and issues around
              multi-threading and vim. The intent is to fit a simple solution into the
              existing structures.

              -ernie

              >
              > For me alone it'll be too much work. Who else would love to join and
              > help fixing this?
              >
              > The main problem is that everything is based on a huge hack:
              > gui_process_events which gets called even when viml is running, so that
              > ctrl-c can be "looked ahead" and viml can be aborted.
              > This gui_process_events like functions has side effects which I think it
              > should not have, see thread linked above about resize which is not used
              > often enough to really care about.
              >
              > Which is the corret way to fix this? Have a separate gtk thread which
              > gets keyboard input, processes it, and when finding a ctrl-c sets
              > PySetInterrupt causing python to stop (running in a different thread).
              >
              > And that's where/why I stopped - I had the feeling that fixing this
              > would require adding multi threading support or such - thus rewriting
              > much gui code. I personally think its the future - but I cannot do it
              > alone. Does anybody have different views / ideas / solutions ?
              >
              > Marc Weber
              >

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
            • Marc Weber
              ... Do you have some spare time left ? I already have patched the interrupt stuff, and it works for the console, but it still does not work for gui, because
              Message 6 of 10 , Sep 5, 2013
                Excerpts from Ernie Rael's message of Thu Sep 05 19:14:35 +0200 2013:
                > I've never contributed to vim, and only occasionally use python (never
                > with vim). I'm not a good candidate. But I really enjoy considering real
                > time issues.
                Do you have some spare time left ?

                I already have patched the interrupt stuff, and it works for the
                console, but it still does not work for gui, because there is no
                terminal sending SIGINT !

                That's why I'm talking about adding multiple threads.

                We could also get rid of all the resize issues inerrupting viml for
                loops and so on.

                However it would be a lot of work probably *and* we need the community
                to help testing.

                Who would test?

                Marc Weber

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              • Ernie Rael
                ... If I saw things correctly, select is run under a timer. One possible solution, the timer interrupt could enable SIGINT interrupt if the command is taking a
                Message 7 of 10 , Sep 5, 2013
                  On 9/3/2013 4:41 PM, James McCoy wrote:
                  > On Tue, Sep 03, 2013 at 03:09:11PM -0700, Ernie Rael wrote:
                  >> It seems that if the SIGINT is always caught/handled and only used
                  >> to set a flag such as: interrupt_pending = TRUE, then vim's main
                  >> loop could do something like
                  >>
                  >> if(interrupt_pending == TRUE) {
                  >> interrupt_pending = FALSE;
                  >> /* do vim's process interrupt stuff */
                  >> }
                  > Something like this would also address the problem where doing something
                  > like ":call taglist('a')" with a large tags file (e.g., all of
                  > /usr/include) takes multiple minutes to return. This happens because
                  > Vim currently uses select() to monitor for the <C-c> keypress while the
                  > tag processing is happening and those calls to select() dominate the
                  > run time.

                  If I saw things correctly, select is run under a timer. One possible
                  solution, the timer interrupt could enable SIGINT interrupt if the
                  command is taking a long time and stop using select. Could wait several
                  hundred milliseconds (or even seconds) before doing it. Maybe the bell
                  could be rung to indicate a long running action.

                  This has some issues, but it might not be too bad. In particular, you
                  don't want to turn on ICANON or anything else that might affect the
                  input character q. Also, while in this mode certain characters (no more
                  than just a couple) will not be put into the typeahead Q. I'm assuming
                  the keyboard chars for abort and suspend will not be put into the Q but
                  at least these particular signals can be ignored. If this is a problem,
                  then I suppose this could be optional behavior (OMG, not another option...)

                  -ernie

                  >
                  > Cheers,

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php

                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                • Marc Weber
                  ... Even if (I didn t check) the timer would not fire because no gtk event loop will be run as long as vim is executing python code ? What is happening
                  Message 8 of 10 , Sep 5, 2013
                    Excerpts from Ernie Rael's message of Thu Sep 05 19:30:37 +0200 2013:
                    > If I saw things correctly, select is run under a timer.
                    Even if (I didn't check) the timer would not fire because no gtk event
                    loop will be run as long as vim is executing python code ?

                    What is happening exactly?
                    While VimL is running vim calls a "run gui event loop" occasionally, to
                    found out that ctrl-c was pressed (otherwise you cannot abort viml).

                    Why does this fail for python? You cannot tell pyton "let me run my
                    event loop occasinally to find out whether ctrl-c was pressed", because
                    you cannot patch the python interpreter implementetation.

                    For this reason I think we need two threads: one for the event loop, one
                    running python. the PyError function I talked about is thread safe (I
                    looked it up)

                    If we get started with this all, we can try to fix other issues as well.
                    This will need da lot of:
                    - work, dedication
                    - support from the community (testing)
                    - documenting the new ctrl-c behaviour

                    If somebody knowing vim better than I do could comment on the "needs
                    thread" thing - eg deny or acknowledege, then I'd know not doing the
                    wrong thing.

                    Marc Weber

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/groups/opt_out.
                  • Ernie Rael
                    ... OK, I went through the old emails again and see that now. ... Are you sure about that? Isn t there still a controlling tty? I m not certain and don t have
                    Message 9 of 10 , Sep 8, 2013
                      On 9/5/2013 10:28 AM, Marc Weber wrote:
                      
                      I already have patched the interrupt stuff, and it works for the
                      console, 
                      OK, I went through the old emails again and see that now.
                      but it still does not work for gui, because there is no
                      terminal sending SIGINT !

                      Are you sure about that? Isn't there still a controlling tty? I'm not certain and don't have a unix system to test with. I thought the big issue was the unreliability of signals in a typical gui environment. This may not be as much of an issue with vim partly because it looks like vim doesn't use the event loop and PySetInterrupt() should be async safe (IIUC); vim only picks up events when it knows there are events to pick up, it doesn't hang in the event loop.

                      Have you tried enabling SIGINT in gui mode? This isn't a general solution for the various problems with the current model, but it might fix the python hang issue.

                      BTW, there's an interesting page on "X and signals" http://www.eng.cam.ac.uk/help/tpl/graphics/X/signals.html

                      
                      That's why I'm talking about adding multiple threads.

                      Certainly seems the right direction.

                      
                      We could also get rid of all the resize issues inerrupting viml for
                      loops and so on.

                      After a quick look I'd aim for introducing USE_INPUT_THREAD define and introduce a lock for the input_buffer and maybe of few other related data structures; the lock is used in the add_to_input_buf(), get_input_buf() and related methods. The input thread would hang in the X event loop. Protecting the input buffer takes care of the key events; there may be other events that can be put into the input queue. Some of the other events are directly executed; a strategy is needed for these. One possibility would be to have a second queue (maybe a priority Q) for directly executed events. It might be useful to have some flags such as ctrl_c_input.

                      In the vim thread have something like
                          gui_mch_update(void)
                          {
                              if(ctrl_c_input) {
                                  got_int = TRUE;
                                  ctrl_c_input = FALSE;
                              }
                              if(other_events_to_handle)
                                  handle_other_events(); /* (in gui.c?) resize, menu, ... */
                          }
                      I don't think there is a reason that the input characters would only be made visible to the vim thread, in gui_mch_update(), but I guess that's possible.

                      Though conceptually simple (?), things are spread out and there are certainly other things to look out for. I don't have any feel for how much work such a change would be.

                      I'm assuming that output would continue to be in the vim thread.

                      To start with, only a single gui (the most commonly used I would hope) needs to be ported. I suspect that the rest would follow quickly. I usually like to start by getting something that works; after that there could probably be simplifications of various areas.

                      Since I'm familiar with neither vim or X code (other than looking through vim over the last week), the first thing I'd ask is if this makes any sense at all. Someone familiar with gui programming might suggest a much better, or more precise, approach.

                      -ernie

                      
                      However it would be a lot of work probably *and* we need the community
                      to help testing.
                      
                      Who would test?
                      
                      Marc Weber
                      
                      

                      --
                      --
                      You received this message from the "vim_dev" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php
                       
                      ---
                      You received this message because you are subscribed to the Google Groups "vim_dev" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                      For more options, visit https://groups.google.com/groups/opt_out.
                    • Marc Weber
                      ... No. If you use --nofork and if you press ctrl-c in the terminal you might be lucky. However this is not enough because - people start vim from menu -
                      Message 10 of 10 , Sep 8, 2013
                        > Are you sure about that?
                        No. If you use --nofork and if you press ctrl-c in the terminal you
                        might be lucky.

                        However this is not enough because
                        - people start vim from menu
                        - people might not use --nofork

                        Current implementation does not interrupt anything using but VimL even
                        using --nofork (tested pyton and ruby)

                        > Have you tried enabling SIGINT in gui mode?
                        There is a problem: Its hard to know the pid unless you take notes
                        upfront. Maybe you can send such a signal, but users are used to
                        pressing "ctrl-c" in whatever window is active?

                        > Though conceptually simple (?), things are spread out and there are
                        > certainly other things to look out for. I don't have any feel for how
                        > much work such a change would be.
                        A lot, but having nice async support is something we should strive for
                        :(

                        I cannot afford hacking on Vim for days at the moment. So this has to
                        wait in any case unless somebody is willing to fund such work.

                        I'd also like to see at least 5 people wanting this to be fixed.
                        We're two now.

                        Marc Weber

                        --
                        --
                        You received this message from the "vim_dev" maillist.
                        Do not top-post! Type your reply below the text you are replying to.
                        For more information, visit http://www.vim.org/maillist.php

                        ---
                        You received this message because you are subscribed to the Google Groups "vim_dev" group.
                        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                        For more options, visit https://groups.google.com/groups/opt_out.
                      Your message has been successfully submitted and would be delivered to recipients shortly.