Loading ...
Sorry, an error occurred while loading the content.

Vim SEGV after netbeans messes up syntax data structures

Expand Messages
  • Xavier de Gaye
    ... Vim version 7.1.298 on linux. Vim crashes processing received netbeans messages while redrawing the windows. This happens about once every five times when
    Message 1 of 6 , May 25, 2008
    • 0 Attachment
      Problem:
      --------
      Vim version 7.1.298 on linux.
      Vim crashes processing received netbeans messages while redrawing the
      windows. This happens about once every five times when running the
      pyclewn test case described at the end of this mail. There is also
      another case where Vim crashes, with clewn this time, when fast stepping
      (pressing the 'S' key continuously, causing the gdb 'step' command to
      run continuously). The crash happens with the clewn case after about 20
      seconds and the gdb backtrace is the same as in the pyclewn test case.

      The gdb backtrace with the pyclewn test is:

      #0 0xffffe410 in __kernel_vsyscall ()
      #1 0xb7973c76 in kill () from /lib/tls/libc.so.6
      #2 0x081443b6 in may_core_dump () at os_unix.c:3024
      #3 0x0814434a in mch_exit (r=1) at os_unix.c:2989
      #4 0x080ec128 in getout (exitval=1) at main.c:1342
      #5 0x0810f965 in preserve_exit () at misc1.c:8370
      #6 0x08142802 in deathtrap (sigarg=11) at os_unix.c:1048
      #7 <signal handler called>
      #8 0x081940c4 in in_id_list (cur_si=0x83733c0, list=0x74,
      ssp=0x8387e7c, contained=0) at syntax.c:5863
      #9 0x0818d504 in syn_current_attr (syncing=0, displaying=1,
      can_spell=0x0, keep_state=0) at syntax.c:1975
      #10 0x0818cfe5 in get_syntax_attr (col=8, can_spell=0x0, keep_state=0)
      at syntax.c:1771
      #11 0x0815ebdb in win_line (wp=0x820ebf0, lnum=8, startrow=7,
      endrow=26, nochange=1) at screen.c:3896
      #12 0x0815b989 in win_update (wp=0x820ebf0) at screen.c:1765
      #13 0x08159d08 in update_screen (type=40) at screen.c:522
      #14 0x080ebd91 in main_loop (cmdwin=0, noexmode=0) at main.c:1110
      #15 0x080eba46 in main (argc=7, argv=0xbfb17254) at main.c:940

      Gdb debug session:
      ------------------
      Program received signal SIGSEGV, Segmentation fault.
      0x081940c4 in in_id_list (cur_si=0x8362b78, list=0x74, ssp=0x8386ff4,
      contained=0) at syntax.c:5863
      (gdb) print current_state
      $3 = {ga_len = 1, ga_maxlen = 6, ga_itemsize = 80, ga_growsize = 3,
      ga_data = 0x8382820}

      Note that the following invariant is false (hence the crash):
      ga_data <= cur_si < ga_data + 6 * 80

      Valgrind session:
      -----------------
      When pyclewn is instrumented to run valgrind on top of Vim (see test
      case below), the following memory error reported by Valgrind occurs
      systematically when running the pyclewn test case (but not causing a
      SEGV each time).
      It shows that the netbeans callback (invoked through ui_breakcheck)
      modifies the syntax.c 'current_state' data structure by calling
      invalidate_current_state() under the feet of syn_current_attr().

      ==4908== Invalid read of size 4
      ==4908== at 0x818D4F9: syn_current_attr (syntax.c:1975)
      ==4908== by 0x818CFE4: get_syntax_attr (syntax.c:1771)
      ==4908== by 0x815EBDA: win_line (screen.c:3896)
      ==4908== by 0x815B988: win_update (screen.c:1765)
      ==4908== by 0x8159D07: update_screen (screen.c:522)
      ==4908== by 0x80EBD90: main_loop (main.c:1110)
      ==4908== by 0x80EBA45: main (main.c:940)
      ==4908== by 0x1BE4FE22: __libc_start_main (in /lib/tls/libc-2.3.6.so)
      ==4908== by 0x806B12C: (within
      /home/xavier/src/packages/vim/vim_svn/vim7/src/vim)
      ==4908== Address 0x1C604594 is 68 bytes inside a block of size 240 free'd
      ==4908== at 0x1B901382: free (vg_replace_malloc.c:235)
      ==4908== by 0x8111FE1: vim_free (misc2.c:1584)
      ==4908== by 0x8112221: ga_clear (misc2.c:1881)
      ==4908== by 0x818B5BF: clear_current_state (syntax.c:660)
      ==4908== by 0x818CD57: invalidate_current_state (syntax.c:1615)
      ==4908== by 0x818B255: syntax_start (syntax.c:517)
      ==4908== by 0x815CCC7: win_line (screen.c:2719)
      ==4908== by 0x815B988: win_update (screen.c:1765)
      ==4908== by 0x8159D07: update_screen (screen.c:522)
      ==4908== by 0x81C92D5: nb_do_cmd (netbeans.c:2079)
      ==4908== by 0x81C7288: nb_parse_cmd (netbeans.c:839)
      ==4908== by 0x81C6FBE: nb_parse_messages (netbeans.c:667)
      ==4908== by 0x81C710D: messageFromNetbeans (netbeans.c:754)
      ==4908== by 0x1BBE8872: (within /usr/lib/libgdk-x11-2.0.so.0.400.14)
      ==4908== by 0x1BD5C9C6: (within /usr/lib/libglib-2.0.so.0.400.8)
      ==4908== by 0x1BD387BA: g_main_context_dispatch (in
      /usr/lib/libglib-2.0.so.0.400.8)
      ==4908== by 0x1BD3A241: (within
      /usr/lib/libglib-2.0.so.0.400.8)==4908== by 0x1BD3A727:
      g_main_context_iteration (in /usr/lib/libglib-2.0.so.0.400.8)
      ==4908== by 0x1BA2671C: gtk_main_iteration_do (in
      /usr/lib/libgtk-x11-2.0.so.0.400.14)
      ==4908== by 0x81C2651: gui_mch_update (gui_gtk_x11.c:6382)
      ==4908== by 0x81A33A2: ui_breakcheck (ui.c:365)
      ==4908== by 0x810F9EA: fast_breakcheck (misc1.c:8423)
      ==4908== by 0x81544B1: regmatch (regexp.c:3788)
      ==4908== by 0x8154249: regtry (regexp.c:3656)
      ==4908== by 0x815406B: vim_regexec_both (regexp.c:3545)
      ==4908== by 0x8153B67: vim_regexec_multi (regexp.c:3354)
      ==4908== by 0x818EF79: syn_regexec (syntax.c:3100)
      ==4908== by 0x818D591: syn_current_attr (syntax.c:2002)
      ==4908== by 0x818CFE4: get_syntax_attr (syntax.c:1771)
      ==4908== by 0x815EBDA: win_line (screen.c:3896)
      ==4908== by 0x815B988: win_update (screen.c:1765)
      ==4908== by 0x8159D07: update_screen (screen.c:522)
      ==4908== by 0x80EBD90: main_loop (main.c:1110)
      ==4908== by 0x80EBA45: main (main.c:940)
      ==4908== by 0x1BE4FE22: __libc_start_main (in /lib/tls/libc-2.3.6.so)
      ==4908== by 0x806B12C: (within
      /home/xavier/src/packages/vim/vim_svn/vim7/src/vim)

      Solution:
      ---------
      See patch below (also attached to this mail as vim_callback.diff).
      Implement a global flag named 'callback_restricted' that restrict the
      use of Vim functions by the gdk callbacks to the scope of the
      normal_cmd() function within the main loop (where it is considered to be
      safe to use the full set of Vim functions).

      The following tests have been completed successfully with this patch:
      * no Vim crash with the pyclewn test case
      * no error reported by Valgrind with the pyclewn test case
      * Vim does not crash when doing fast stepping with clewn
      * netbeans does process messages when Vim is idle in normal_cmd()
      * the full pyclewn regression test suite runs Ok.

      Note that attempting to prevent netbeans to recursively call win_update
      is not good enough. In that case more memory errors occur after netbeans
      call buf_jump_open_win or set_curbuf.

      === patch -d vim7 -p0 < vim_callback.diff =========================
      Index: src/netbeans.c
      ===================================================================
      --- src/netbeans.c (revision 1011)
      +++ src/netbeans.c (working copy)
      @@ -749,8 +749,11 @@
      return; /* don't try to parse it */
      }

      - /* Parse the messages, but avoid recursion. */
      - if (level == 1)
      + /* Parse the messages, but avoid recursion
      + * and do not process messages when the use
      + * of vim functions is restricted within callbacks.
      + */
      + if (level == 1 && ! callback_restricted)
      nb_parse_messages();

      --level;
      Index: src/main.c
      ===================================================================
      --- src/main.c (revision 1011)
      +++ src/main.c (working copy)
      @@ -1178,7 +1178,19 @@
      do_exmode(exmode_active == EXMODE_VIM);
      }
      else
      + {
      +#ifdef FEAT_NETBEANS_INTG
      + /* Allow full use of vim functions in callbacks,
      + * but not within the command-line window or in
      + * ":global/path/visual" commands.*/
      + if (!cmdwin && !noexmode)
      + callback_restricted = FALSE;
      +#endif
      normal_cmd(&oa, TRUE);
      +#ifdef FEAT_NETBEANS_INTG
      + callback_restricted = TRUE;
      +#endif
      + }
      }
      }

      Index: src/globals.h
      ===================================================================
      --- src/globals.h (revision 1011)
      +++ src/globals.h (working copy)
      @@ -1008,6 +1008,11 @@
      /* set after swap write error msg */
      EXTERN int undo_off INIT(= FALSE); /* undo switched off for now */
      EXTERN int global_busy INIT(= 0); /* set when :global is executing */
      +#ifdef FEAT_NETBEANS_INTG
      +EXTERN int callback_restricted INIT(= TRUE);/* callbacks have a restricted
      + * use of vim functions when
      + * this flag is set */
      +#endif
      EXTERN int listcmd_busy INIT(= FALSE); /* set when :argdo, :windo or
      :bufdo is executing */
      EXTERN int need_start_insertmode INIT(= FALSE);
      ===========================================================


      Test case:
      ----------
      Get pyclewn from the subversion repository
      (see http://pyclewn.wiki.sourceforge.net/svn+).
      This is currently pyclewn subversion revision 13.
      Install pyclewn
      Run pyclewn from the pyclewn distribution directory:
      ./pyclewn -l nbdebug -f logfile --gdb=async
      Once pyclewn is started, run the following Vim commands:
      :Cmapkeys | Cfile testsuite/foobar | Cbreak main | Crun
      CTRL-W W
      :q
      S
      Vim crashes on average once every five times (the 'S' key must be hit
      quickly after ':q').

      To use Valgrind, the pyclewn code must be instrumented:
      * edit clewn/dispatcher.py and search for 'valgrind'
      * uncomment the two lines that enable valgrind
      * you need also to insert before the two uncommented lines, the
      following two lines, to increase valgring stack depth:
      args[:0] = ["--num-callers=50"]
      args[:0] = ["--show-below-main=yes"]
      The test is run as above with the same Vim commands, except that pyclewn
      is started without logs and that you don't need to be fast when typing
      the last two Vim commands, the error is systematic:
      ./pyclewn --gdb=async


      -- Xavier
      Les Chemins de Lokoti -- http://lokoti.alwaysdata.net

      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_dev" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • Bram Moolenaar
      ... This will help, but I think it s still possible that Vim is updating the screen somewhere when a netbeans event is received. In functions called from
      Message 2 of 6 , May 26, 2008
      • 0 Attachment
        Xavier de Gaye wrote:

        > Problem:
        > --------
        > Vim version 7.1.298 on linux.
        > Vim crashes processing received netbeans messages while redrawing the
        > windows. This happens about once every five times when running the
        > pyclewn test case described at the end of this mail. There is also
        > another case where Vim crashes, with clewn this time, when fast stepping
        > (pressing the 'S' key continuously, causing the gdb 'step' command to
        > run continuously). The crash happens with the clewn case after about 20
        > seconds and the gdb backtrace is the same as in the pyclewn test case.
        >
        > The gdb backtrace with the pyclewn test is:
        >
        > #0 0xffffe410 in __kernel_vsyscall ()
        > #1 0xb7973c76 in kill () from /lib/tls/libc.so.6
        > #2 0x081443b6 in may_core_dump () at os_unix.c:3024
        > #3 0x0814434a in mch_exit (r=1) at os_unix.c:2989
        > #4 0x080ec128 in getout (exitval=1) at main.c:1342
        > #5 0x0810f965 in preserve_exit () at misc1.c:8370
        > #6 0x08142802 in deathtrap (sigarg=11) at os_unix.c:1048
        > #7 <signal handler called>
        > #8 0x081940c4 in in_id_list (cur_si=0x83733c0, list=0x74,
        > ssp=0x8387e7c, contained=0) at syntax.c:5863
        > #9 0x0818d504 in syn_current_attr (syncing=0, displaying=1,
        > can_spell=0x0, keep_state=0) at syntax.c:1975
        > #10 0x0818cfe5 in get_syntax_attr (col=8, can_spell=0x0, keep_state=0)
        > at syntax.c:1771
        > #11 0x0815ebdb in win_line (wp=0x820ebf0, lnum=8, startrow=7,
        > endrow=26, nochange=1) at screen.c:3896
        > #12 0x0815b989 in win_update (wp=0x820ebf0) at screen.c:1765
        > #13 0x08159d08 in update_screen (type=40) at screen.c:522
        > #14 0x080ebd91 in main_loop (cmdwin=0, noexmode=0) at main.c:1110
        > #15 0x080eba46 in main (argc=7, argv=0xbfb17254) at main.c:940
        >
        > Gdb debug session:
        > ------------------
        > Program received signal SIGSEGV, Segmentation fault.
        > 0x081940c4 in in_id_list (cur_si=0x8362b78, list=0x74, ssp=0x8386ff4,
        > contained=0) at syntax.c:5863
        > (gdb) print current_state
        > $3 = {ga_len = 1, ga_maxlen = 6, ga_itemsize = 80, ga_growsize = 3,
        > ga_data = 0x8382820}
        >
        > Note that the following invariant is false (hence the crash):
        > ga_data <= cur_si < ga_data + 6 * 80
        >
        > Valgrind session:
        > -----------------
        > When pyclewn is instrumented to run valgrind on top of Vim (see test
        > case below), the following memory error reported by Valgrind occurs
        > systematically when running the pyclewn test case (but not causing a
        > SEGV each time).
        > It shows that the netbeans callback (invoked through ui_breakcheck)
        > modifies the syntax.c 'current_state' data structure by calling
        > invalidate_current_state() under the feet of syn_current_attr().
        >
        > ==4908== Invalid read of size 4
        > ==4908== at 0x818D4F9: syn_current_attr (syntax.c:1975)
        > ==4908== by 0x818CFE4: get_syntax_attr (syntax.c:1771)
        > ==4908== by 0x815EBDA: win_line (screen.c:3896)
        > ==4908== by 0x815B988: win_update (screen.c:1765)
        > ==4908== by 0x8159D07: update_screen (screen.c:522)
        > ==4908== by 0x80EBD90: main_loop (main.c:1110)
        > ==4908== by 0x80EBA45: main (main.c:940)
        > ==4908== by 0x1BE4FE22: __libc_start_main (in /lib/tls/libc-2.3.6.so)
        > ==4908== by 0x806B12C: (within
        > /home/xavier/src/packages/vim/vim_svn/vim7/src/vim)
        > ==4908== Address 0x1C604594 is 68 bytes inside a block of size 240 free'd
        > ==4908== at 0x1B901382: free (vg_replace_malloc.c:235)
        > ==4908== by 0x8111FE1: vim_free (misc2.c:1584)
        > ==4908== by 0x8112221: ga_clear (misc2.c:1881)
        > ==4908== by 0x818B5BF: clear_current_state (syntax.c:660)
        > ==4908== by 0x818CD57: invalidate_current_state (syntax.c:1615)
        > ==4908== by 0x818B255: syntax_start (syntax.c:517)
        > ==4908== by 0x815CCC7: win_line (screen.c:2719)
        > ==4908== by 0x815B988: win_update (screen.c:1765)
        > ==4908== by 0x8159D07: update_screen (screen.c:522)
        > ==4908== by 0x81C92D5: nb_do_cmd (netbeans.c:2079)
        > ==4908== by 0x81C7288: nb_parse_cmd (netbeans.c:839)
        > ==4908== by 0x81C6FBE: nb_parse_messages (netbeans.c:667)
        > ==4908== by 0x81C710D: messageFromNetbeans (netbeans.c:754)
        > ==4908== by 0x1BBE8872: (within /usr/lib/libgdk-x11-2.0.so.0.400.14)
        > ==4908== by 0x1BD5C9C6: (within /usr/lib/libglib-2.0.so.0.400.8)
        > ==4908== by 0x1BD387BA: g_main_context_dispatch (in
        > /usr/lib/libglib-2.0.so.0.400.8)
        > ==4908== by 0x1BD3A241: (within
        > /usr/lib/libglib-2.0.so.0.400.8)==4908== by 0x1BD3A727:
        > g_main_context_iteration (in /usr/lib/libglib-2.0.so.0.400.8)
        > ==4908== by 0x1BA2671C: gtk_main_iteration_do (in
        > /usr/lib/libgtk-x11-2.0.so.0.400.14)
        > ==4908== by 0x81C2651: gui_mch_update (gui_gtk_x11.c:6382)
        > ==4908== by 0x81A33A2: ui_breakcheck (ui.c:365)
        > ==4908== by 0x810F9EA: fast_breakcheck (misc1.c:8423)
        > ==4908== by 0x81544B1: regmatch (regexp.c:3788)
        > ==4908== by 0x8154249: regtry (regexp.c:3656)
        > ==4908== by 0x815406B: vim_regexec_both (regexp.c:3545)
        > ==4908== by 0x8153B67: vim_regexec_multi (regexp.c:3354)
        > ==4908== by 0x818EF79: syn_regexec (syntax.c:3100)
        > ==4908== by 0x818D591: syn_current_attr (syntax.c:2002)
        > ==4908== by 0x818CFE4: get_syntax_attr (syntax.c:1771)
        > ==4908== by 0x815EBDA: win_line (screen.c:3896)
        > ==4908== by 0x815B988: win_update (screen.c:1765)
        > ==4908== by 0x8159D07: update_screen (screen.c:522)
        > ==4908== by 0x80EBD90: main_loop (main.c:1110)
        > ==4908== by 0x80EBA45: main (main.c:940)
        > ==4908== by 0x1BE4FE22: __libc_start_main (in /lib/tls/libc-2.3.6.so)
        > ==4908== by 0x806B12C: (within
        > /home/xavier/src/packages/vim/vim_svn/vim7/src/vim)
        >
        > Solution:
        > ---------
        > See patch below (also attached to this mail as vim_callback.diff).
        > Implement a global flag named 'callback_restricted' that restrict the
        > use of Vim functions by the gdk callbacks to the scope of the
        > normal_cmd() function within the main loop (where it is considered to be
        > safe to use the full set of Vim functions).
        >
        > The following tests have been completed successfully with this patch:
        > * no Vim crash with the pyclewn test case
        > * no error reported by Valgrind with the pyclewn test case
        > * Vim does not crash when doing fast stepping with clewn
        > * netbeans does process messages when Vim is idle in normal_cmd()
        > * the full pyclewn regression test suite runs Ok.
        >
        > Note that attempting to prevent netbeans to recursively call win_update
        > is not good enough. In that case more memory errors occur after netbeans
        > call buf_jump_open_win or set_curbuf.

        This will help, but I think it's still possible that Vim is updating the
        screen somewhere when a netbeans event is received. In functions called
        from normal_cmd() it's very well possible that the screen is updated
        directly instead of setting the must_redraw flag.

        One way to solve this is to set that callback_restricted flag in
        ui_breakcheck(). Because we don't want to handle the netbeans messages
        while checking if the user typed CTRL-C.

        It might still not be sufficient though. It's difficult to overview
        what happens with these asynchronous messages. And we also have to be
        careful not to block event handling (I think that happens when you would
        use the may_garbage_collect flag).

        Perhaps you can try out the change to ui_breakcheck() and tell us what
        happens. Perhaps there are other places where gui_mch_update() is
        called and we don't want to handle events, thus setting the flag in
        gui_mch_update() might be an alternative. But you can't do that
        always...

        --
        A M00se once bit my sister ...
        "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

        /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
        /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
        \\\ download, build and distribute -- http://www.A-A-P.org ///
        \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_dev" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • Xavier de Gaye
        ... Setting the callback_restricted flag in ui_breakcheck() does not work. Most pyclewn automated tests fail because the netbeans messages are received (and
        Message 3 of 6 , May 31, 2008
        • 0 Attachment
          On Mon, May 26, 2008 at 9:24 PM, Bram Moolenaar <Bram@...> wrote:
          > Xavier de Gaye wrote:
          >> Problem:
          >> --------
          >> Vim version 7.1.298 on linux.
          >> Vim crashes processing received netbeans messages while redrawing the
          >> windows. This happens about once every five times when running the
          >> [...]
          >> The following tests have been completed successfully with this patch:
          >> * no Vim crash with the pyclewn test case
          >> * no error reported by Valgrind with the pyclewn test case
          >> * Vim does not crash when doing fast stepping with clewn
          >> * netbeans does process messages when Vim is idle in normal_cmd()
          >> * the full pyclewn regression test suite runs Ok.
          >> [...]
          >
          > This will help, but I think it's still possible that Vim is updating the
          > screen somewhere when a netbeans event is received. In functions called
          > from normal_cmd() it's very well possible that the screen is updated
          > directly instead of setting the must_redraw flag.
          >
          > One way to solve this is to set that callback_restricted flag in
          > ui_breakcheck(). Because we don't want to handle the netbeans messages
          > while checking if the user typed CTRL-C.
          >
          > It might still not be sufficient though. It's difficult to overview
          > what happens with these asynchronous messages. And we also have to be
          > careful not to block event handling (I think that happens when you would
          > use the may_garbage_collect flag).
          >
          > Perhaps you can try out the change to ui_breakcheck() and tell us what
          > happens. Perhaps there are other places where gui_mch_update() is
          > called and we don't want to handle events, thus setting the flag in
          > gui_mch_update() might be an alternative. But you can't do that
          > always...

          Setting the callback_restricted flag in ui_breakcheck() does not work.
          Most pyclewn automated tests fail because the netbeans messages are
          received (and queued) in ui_breakcheck() and nb_parse_messages() never
          gets a chance to process the received messages. This points out to
          a problem that may occur when netbeans is run interactively: if the
          last netbeans message is received in ui_breakcheck(), it is not
          processed by nb_parse_messages() and appears to the user as not
          received at all (until the next message arrives).
          Except for that problem, this solution fixes the memory corruption as
          expected in all the tests that do run.

          Another way to fix this is to prevent the netbeans callback to call
          any vim function at all. This cancels the need of a global
          callback_restricted flag. The callback queues the received messages
          and exit the gui event loop. The messages are processed in the idle
          loop where vim is waiting for character input.

          The following patch (also attached to this mail) implements this
          solution for FEAT_GUI_GTK. All the tests are successful (see the list
          of tests above, in the original post) including the 51 pyclewn tests.

          I think it is possible to implement the same solution for
          FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
          don't know what application can be used to test the patch (maybe Agide ?).


          === patch -d vim7 -p0 < vim_callback_2.diff =======================
          Index: src/netbeans.c
          ===================================================================
          --- src/netbeans.c (revision 1024)
          +++ src/netbeans.c (working copy)
          @@ -617,13 +617,13 @@
          /*
          * While there's still a command in the work queue, parse and execute it.
          */
          - static void
          -nb_parse_messages(void)
          + void
          +netbeans_parse_messages(void)
          {
          char_u *p;
          queue_T *node;

          - while (head.next != &head)
          + while (head.next != NULL && head.next != &head)
          {
          node = head.next;

          @@ -707,7 +707,9 @@
          static char_u *buf = NULL;
          int len;
          int readlen = 0;
          +#ifndef FEAT_GUI_GTK
          static int level = 0;
          +#endif

          if (sd < 0)
          {
          @@ -715,7 +717,9 @@
          return;
          }

          +#ifndef FEAT_GUI_GTK
          ++level; /* recursion guard; this will be called from the X event loop */
          +#endif

          /* Allocate a buffer to read into. */
          if (buf == NULL)
          @@ -749,11 +753,16 @@
          return; /* don't try to parse it */
          }

          +#ifdef FEAT_GUI_GTK
          + if (gtk_main_level() > 0)
          + gtk_main_quit();
          +#else
          /* Parse the messages, but avoid recursion. */
          if (level == 1)
          - nb_parse_messages();
          + netbeans_parse_messages();

          --level;
          +#endif
          }

          /*
          Index: src/proto/netbeans.pro
          ===================================================================
          --- src/proto/netbeans.pro (revision 1024)
          +++ src/proto/netbeans.pro (working copy)
          @@ -22,4 +22,5 @@
          void netbeans_draw_multisign_indicator __ARGS((int row));
          void netbeans_draw_multisign_indicator __ARGS((int row));
          void netbeans_gutter_click __ARGS((linenr_T lnum));
          +void netbeans_parse_messages __ARGS((void));
          /* vim: set ft=c : */
          Index: src/getchar.c
          ===================================================================
          --- src/getchar.c (revision 1024)
          +++ src/getchar.c (working copy)
          @@ -2883,6 +2883,10 @@
          #endif
          )
          {
          +
          + /* Process the queued netbeans messages. */
          + netbeans_parse_messages();
          +
          if (got_int || (script_char = getc(scriptin[curscript])) < 0)
          {
          /* Reached EOF.
          Index: src/gui_gtk_x11.c
          ===================================================================
          --- src/gui_gtk_x11.c (revision 1024)
          +++ src/gui_gtk_x11.c (working copy)
          @@ -6476,6 +6476,9 @@
          focus = gui.in_focus;
          }

          + /* Process the queued netbeans messages. */
          + netbeans_parse_messages();
          +
          /*
          * Loop in GTK+ processing until a timeout or input occurs.
          * Skip this if input is available anyway (can happen in rare
          ===================================================================


          -- Xavier
          Les Chemins de Lokoti -- http://lokoti.alwaysdata.net

          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_dev" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Bram Moolenaar
          ... This makes sense. I ll put this in the todo list for having a closer look. ... This should not be changed without proper testing. So let s leave it as-is
          Message 4 of 6 , Jun 1, 2008
          • 0 Attachment
            Xavier de Gaye wrote:

            > >> Problem:
            > >> --------
            > >> Vim version 7.1.298 on linux.
            > >> Vim crashes processing received netbeans messages while redrawing the
            > >> windows. This happens about once every five times when running the
            > >> [...]
            > >> The following tests have been completed successfully with this patch:
            > >> * no Vim crash with the pyclewn test case
            > >> * no error reported by Valgrind with the pyclewn test case
            > >> * Vim does not crash when doing fast stepping with clewn
            > >> * netbeans does process messages when Vim is idle in normal_cmd()
            > >> * the full pyclewn regression test suite runs Ok.
            > >> [...]
            > >
            > > This will help, but I think it's still possible that Vim is updating the
            > > screen somewhere when a netbeans event is received. In functions called
            > > from normal_cmd() it's very well possible that the screen is updated
            > > directly instead of setting the must_redraw flag.
            > >
            > > One way to solve this is to set that callback_restricted flag in
            > > ui_breakcheck(). Because we don't want to handle the netbeans messages
            > > while checking if the user typed CTRL-C.
            > >
            > > It might still not be sufficient though. It's difficult to overview
            > > what happens with these asynchronous messages. And we also have to be
            > > careful not to block event handling (I think that happens when you would
            > > use the may_garbage_collect flag).
            > >
            > > Perhaps you can try out the change to ui_breakcheck() and tell us what
            > > happens. Perhaps there are other places where gui_mch_update() is
            > > called and we don't want to handle events, thus setting the flag in
            > > gui_mch_update() might be an alternative. But you can't do that
            > > always...
            >
            > Setting the callback_restricted flag in ui_breakcheck() does not work.
            > Most pyclewn automated tests fail because the netbeans messages are
            > received (and queued) in ui_breakcheck() and nb_parse_messages() never
            > gets a chance to process the received messages. This points out to
            > a problem that may occur when netbeans is run interactively: if the
            > last netbeans message is received in ui_breakcheck(), it is not
            > processed by nb_parse_messages() and appears to the user as not
            > received at all (until the next message arrives).
            > Except for that problem, this solution fixes the memory corruption as
            > expected in all the tests that do run.
            >
            > Another way to fix this is to prevent the netbeans callback to call
            > any vim function at all. This cancels the need of a global
            > callback_restricted flag. The callback queues the received messages
            > and exit the gui event loop. The messages are processed in the idle
            > loop where vim is waiting for character input.
            >
            > The following patch (also attached to this mail) implements this
            > solution for FEAT_GUI_GTK. All the tests are successful (see the list
            > of tests above, in the original post) including the 51 pyclewn tests.

            This makes sense. I'll put this in the todo list for having a closer
            look.

            > I think it is possible to implement the same solution for
            > FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
            > don't know what application can be used to test the patch (maybe Agide ?).

            This should not be changed without proper testing. So let's leave it
            as-is until we can test it.

            --
            Never under any circumstances take a sleeping pill
            and a laxative on the same night.

            /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
            /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
            \\\ download, build and distribute -- http://www.A-A-P.org ///
            \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_dev" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          • Xavier de Gaye
            ... The problem of Netbeans events are handled while updating the screen, causing a crash has been fixed in Vim 7.2 for gtk. The attached patch to Vim 7.2.55
            Message 5 of 6 , Nov 29, 2008
            • 0 Attachment
              On Sun, Jun 1, 2008 at 12:05 PM, Bram Moolenaar <Bram@...> wrote:
              >
              > Xavier de Gaye wrote:
              >
              >> >> Problem:
              >> >> --------
              >> >> Vim version 7.1.298 on linux.
              >> >> Vim crashes processing received netbeans messages while redrawing the
              >> >> windows. This happens about once every five times when running the
              >> >> [...]
              >> >> The following tests have been completed successfully with this patch:
              >> >> * no Vim crash with the pyclewn test case
              >> >> * no error reported by Valgrind with the pyclewn test case
              >> >> * Vim does not crash when doing fast stepping with clewn
              >> >> * netbeans does process messages when Vim is idle in normal_cmd()
              >> >> * the full pyclewn regression test suite runs Ok.
              >> >> [...]
              >> >
              >> > This will help, but I think it's still possible that Vim is updating the
              >> > screen somewhere when a netbeans event is received. In functions called
              >> > from normal_cmd() it's very well possible that the screen is updated
              >> > directly instead of setting the must_redraw flag.
              >> >
              >> > One way to solve this is to set that callback_restricted flag in
              >> > ui_breakcheck(). Because we don't want to handle the netbeans messages
              >> > while checking if the user typed CTRL-C.
              >> >
              >> > It might still not be sufficient though. It's difficult to overview
              >> > what happens with these asynchronous messages. And we also have to be
              >> > careful not to block event handling (I think that happens when you would
              >> > use the may_garbage_collect flag).
              >> >
              >> > Perhaps you can try out the change to ui_breakcheck() and tell us what
              >> > happens. Perhaps there are other places where gui_mch_update() is
              >> > called and we don't want to handle events, thus setting the flag in
              >> > gui_mch_update() might be an alternative. But you can't do that
              >> > always...
              >>
              >> Setting the callback_restricted flag in ui_breakcheck() does not work.
              >> Most pyclewn automated tests fail because the netbeans messages are
              >> received (and queued) in ui_breakcheck() and nb_parse_messages() never
              >> gets a chance to process the received messages. This points out to
              >> a problem that may occur when netbeans is run interactively: if the
              >> last netbeans message is received in ui_breakcheck(), it is not
              >> processed by nb_parse_messages() and appears to the user as not
              >> received at all (until the next message arrives).
              >> Except for that problem, this solution fixes the memory corruption as
              >> expected in all the tests that do run.
              >>
              >> Another way to fix this is to prevent the netbeans callback to call
              >> any vim function at all. This cancels the need of a global
              >> callback_restricted flag. The callback queues the received messages
              >> and exit the gui event loop. The messages are processed in the idle
              >> loop where vim is waiting for character input.
              >>
              >> The following patch (also attached to this mail) implements this
              >> solution for FEAT_GUI_GTK. All the tests are successful (see the list
              >> of tests above, in the original post) including the 51 pyclewn tests.
              >
              > This makes sense. I'll put this in the todo list for having a closer
              > look.
              >
              >> I think it is possible to implement the same solution for
              >> FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
              >> don't know what application can be used to test the patch (maybe Agide ?).
              >
              > This should not be changed without proper testing. So let's leave it
              > as-is until we can test it.


              The problem of "Netbeans events are handled while updating the screen,
              causing a crash" has been fixed in Vim 7.2 for gtk. The attached patch
              to Vim 7.2.55 (subversion revision 1273) fixes this same problem for
              Windows.

              The following tests have been completed successfully with this patch
              and pyclewn version 0.6 on Windows:
              * Vim processes netbeans messages when idle
              * the full pyclewn regression test suite runs Ok

              Note that the crash could not be reproduced with Vim 7.2 and pyclewn,
              as the current implementation of pyclewn on Windows is too slow.

              Xavier

              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_dev" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            • Bram Moolenaar
              ... Thanks for the patch. Looks OK to me, although I can t try it out. I ll include it. -- This is the polymorph virus! Follow these instructions carefully:
              Message 6 of 6 , Nov 30, 2008
              • 0 Attachment
                Xavier de Gaye wrote:

                > >> >> Problem:
                > >> >> --------
                > >> >> Vim version 7.1.298 on linux.
                > >> >> Vim crashes processing received netbeans messages while redrawing the
                > >> >> windows. This happens about once every five times when running the
                > >> >> [...]
                > >> >> The following tests have been completed successfully with this patch:
                > >> >> * no Vim crash with the pyclewn test case
                > >> >> * no error reported by Valgrind with the pyclewn test case
                > >> >> * Vim does not crash when doing fast stepping with clewn
                > >> >> * netbeans does process messages when Vim is idle in normal_cmd()
                > >> >> * the full pyclewn regression test suite runs Ok.
                > >> >> [...]
                > >> >
                > >> > This will help, but I think it's still possible that Vim is updating the
                > >> > screen somewhere when a netbeans event is received. In functions called
                > >> > from normal_cmd() it's very well possible that the screen is updated
                > >> > directly instead of setting the must_redraw flag.
                > >> >
                > >> > One way to solve this is to set that callback_restricted flag in
                > >> > ui_breakcheck(). Because we don't want to handle the netbeans messages
                > >> > while checking if the user typed CTRL-C.
                > >> >
                > >> > It might still not be sufficient though. It's difficult to overview
                > >> > what happens with these asynchronous messages. And we also have to be
                > >> > careful not to block event handling (I think that happens when you would
                > >> > use the may_garbage_collect flag).
                > >> >
                > >> > Perhaps you can try out the change to ui_breakcheck() and tell us what
                > >> > happens. Perhaps there are other places where gui_mch_update() is
                > >> > called and we don't want to handle events, thus setting the flag in
                > >> > gui_mch_update() might be an alternative. But you can't do that
                > >> > always...
                > >>
                > >> Setting the callback_restricted flag in ui_breakcheck() does not work.
                > >> Most pyclewn automated tests fail because the netbeans messages are
                > >> received (and queued) in ui_breakcheck() and nb_parse_messages() never
                > >> gets a chance to process the received messages. This points out to
                > >> a problem that may occur when netbeans is run interactively: if the
                > >> last netbeans message is received in ui_breakcheck(), it is not
                > >> processed by nb_parse_messages() and appears to the user as not
                > >> received at all (until the next message arrives).
                > >> Except for that problem, this solution fixes the memory corruption as
                > >> expected in all the tests that do run.
                > >>
                > >> Another way to fix this is to prevent the netbeans callback to call
                > >> any vim function at all. This cancels the need of a global
                > >> callback_restricted flag. The callback queues the received messages
                > >> and exit the gui event loop. The messages are processed in the idle
                > >> loop where vim is waiting for character input.
                > >>
                > >> The following patch (also attached to this mail) implements this
                > >> solution for FEAT_GUI_GTK. All the tests are successful (see the list
                > >> of tests above, in the original post) including the 51 pyclewn tests.
                > >
                > > This makes sense. I'll put this in the todo list for having a closer
                > > look.
                > >
                > >> I think it is possible to implement the same solution for
                > >> FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
                > >> don't know what application can be used to test the patch (maybe Agide ?).
                > >
                > > This should not be changed without proper testing. So let's leave it
                > > as-is until we can test it.
                >
                >
                > The problem of "Netbeans events are handled while updating the screen,
                > causing a crash" has been fixed in Vim 7.2 for gtk. The attached patch
                > to Vim 7.2.55 (subversion revision 1273) fixes this same problem for
                > Windows.
                >
                > The following tests have been completed successfully with this patch
                > and pyclewn version 0.6 on Windows:
                > * Vim processes netbeans messages when idle
                > * the full pyclewn regression test suite runs Ok
                >
                > Note that the crash could not be reproduced with Vim 7.2 and pyclewn,
                > as the current implementation of pyclewn on Windows is too slow.

                Thanks for the patch. Looks OK to me, although I can't try it out.
                I'll include it.

                --
                This is the polymorph virus! Follow these instructions carefully:
                1. Send this message to everybody you know.
                2. Format your harddisk.
                Thank you for your cooperation in spreading the most powerful virus ever!

                /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                \\\ download, build and distribute -- http://www.A-A-P.org ///
                \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

                --~--~---------~--~----~------------~-------~--~----~
                You received this message from the "vim_dev" maillist.
                For more information, visit http://www.vim.org/maillist.php
                -~----------~----~----~----~------~----~------~--~---
              Your message has been successfully submitted and would be delivered to recipients shortly.