Loading ...
Sorry, an error occurred while loading the content.

Re: Vim SEGV after netbeans messes up syntax data structures

Expand Messages
  • Xavier de Gaye
    ... Setting the callback_restricted flag in ui_breakcheck() does not work. Most pyclewn automated tests fail because the netbeans messages are received (and
    Message 1 of 6 , May 31, 2008
    • 0 Attachment
      On Mon, May 26, 2008 at 9:24 PM, Bram Moolenaar <Bram@...> wrote:
      > Xavier de Gaye wrote:
      >> Problem:
      >> --------
      >> Vim version 7.1.298 on linux.
      >> Vim crashes processing received netbeans messages while redrawing the
      >> windows. This happens about once every five times when running the
      >> [...]
      >> The following tests have been completed successfully with this patch:
      >> * no Vim crash with the pyclewn test case
      >> * no error reported by Valgrind with the pyclewn test case
      >> * Vim does not crash when doing fast stepping with clewn
      >> * netbeans does process messages when Vim is idle in normal_cmd()
      >> * the full pyclewn regression test suite runs Ok.
      >> [...]
      >
      > This will help, but I think it's still possible that Vim is updating the
      > screen somewhere when a netbeans event is received. In functions called
      > from normal_cmd() it's very well possible that the screen is updated
      > directly instead of setting the must_redraw flag.
      >
      > One way to solve this is to set that callback_restricted flag in
      > ui_breakcheck(). Because we don't want to handle the netbeans messages
      > while checking if the user typed CTRL-C.
      >
      > It might still not be sufficient though. It's difficult to overview
      > what happens with these asynchronous messages. And we also have to be
      > careful not to block event handling (I think that happens when you would
      > use the may_garbage_collect flag).
      >
      > Perhaps you can try out the change to ui_breakcheck() and tell us what
      > happens. Perhaps there are other places where gui_mch_update() is
      > called and we don't want to handle events, thus setting the flag in
      > gui_mch_update() might be an alternative. But you can't do that
      > always...

      Setting the callback_restricted flag in ui_breakcheck() does not work.
      Most pyclewn automated tests fail because the netbeans messages are
      received (and queued) in ui_breakcheck() and nb_parse_messages() never
      gets a chance to process the received messages. This points out to
      a problem that may occur when netbeans is run interactively: if the
      last netbeans message is received in ui_breakcheck(), it is not
      processed by nb_parse_messages() and appears to the user as not
      received at all (until the next message arrives).
      Except for that problem, this solution fixes the memory corruption as
      expected in all the tests that do run.

      Another way to fix this is to prevent the netbeans callback to call
      any vim function at all. This cancels the need of a global
      callback_restricted flag. The callback queues the received messages
      and exit the gui event loop. The messages are processed in the idle
      loop where vim is waiting for character input.

      The following patch (also attached to this mail) implements this
      solution for FEAT_GUI_GTK. All the tests are successful (see the list
      of tests above, in the original post) including the 51 pyclewn tests.

      I think it is possible to implement the same solution for
      FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
      don't know what application can be used to test the patch (maybe Agide ?).


      === patch -d vim7 -p0 < vim_callback_2.diff =======================
      Index: src/netbeans.c
      ===================================================================
      --- src/netbeans.c (revision 1024)
      +++ src/netbeans.c (working copy)
      @@ -617,13 +617,13 @@
      /*
      * While there's still a command in the work queue, parse and execute it.
      */
      - static void
      -nb_parse_messages(void)
      + void
      +netbeans_parse_messages(void)
      {
      char_u *p;
      queue_T *node;

      - while (head.next != &head)
      + while (head.next != NULL && head.next != &head)
      {
      node = head.next;

      @@ -707,7 +707,9 @@
      static char_u *buf = NULL;
      int len;
      int readlen = 0;
      +#ifndef FEAT_GUI_GTK
      static int level = 0;
      +#endif

      if (sd < 0)
      {
      @@ -715,7 +717,9 @@
      return;
      }

      +#ifndef FEAT_GUI_GTK
      ++level; /* recursion guard; this will be called from the X event loop */
      +#endif

      /* Allocate a buffer to read into. */
      if (buf == NULL)
      @@ -749,11 +753,16 @@
      return; /* don't try to parse it */
      }

      +#ifdef FEAT_GUI_GTK
      + if (gtk_main_level() > 0)
      + gtk_main_quit();
      +#else
      /* Parse the messages, but avoid recursion. */
      if (level == 1)
      - nb_parse_messages();
      + netbeans_parse_messages();

      --level;
      +#endif
      }

      /*
      Index: src/proto/netbeans.pro
      ===================================================================
      --- src/proto/netbeans.pro (revision 1024)
      +++ src/proto/netbeans.pro (working copy)
      @@ -22,4 +22,5 @@
      void netbeans_draw_multisign_indicator __ARGS((int row));
      void netbeans_draw_multisign_indicator __ARGS((int row));
      void netbeans_gutter_click __ARGS((linenr_T lnum));
      +void netbeans_parse_messages __ARGS((void));
      /* vim: set ft=c : */
      Index: src/getchar.c
      ===================================================================
      --- src/getchar.c (revision 1024)
      +++ src/getchar.c (working copy)
      @@ -2883,6 +2883,10 @@
      #endif
      )
      {
      +
      + /* Process the queued netbeans messages. */
      + netbeans_parse_messages();
      +
      if (got_int || (script_char = getc(scriptin[curscript])) < 0)
      {
      /* Reached EOF.
      Index: src/gui_gtk_x11.c
      ===================================================================
      --- src/gui_gtk_x11.c (revision 1024)
      +++ src/gui_gtk_x11.c (working copy)
      @@ -6476,6 +6476,9 @@
      focus = gui.in_focus;
      }

      + /* Process the queued netbeans messages. */
      + netbeans_parse_messages();
      +
      /*
      * Loop in GTK+ processing until a timeout or input occurs.
      * Skip this if input is available anyway (can happen in rare
      ===================================================================


      -- Xavier
      Les Chemins de Lokoti -- http://lokoti.alwaysdata.net

      --~--~---------~--~----~------------~-------~--~----~
      You received this message from the "vim_dev" maillist.
      For more information, visit http://www.vim.org/maillist.php
      -~----------~----~----~----~------~----~------~--~---
    • Bram Moolenaar
      ... This makes sense. I ll put this in the todo list for having a closer look. ... This should not be changed without proper testing. So let s leave it as-is
      Message 2 of 6 , Jun 1, 2008
      • 0 Attachment
        Xavier de Gaye wrote:

        > >> Problem:
        > >> --------
        > >> Vim version 7.1.298 on linux.
        > >> Vim crashes processing received netbeans messages while redrawing the
        > >> windows. This happens about once every five times when running the
        > >> [...]
        > >> The following tests have been completed successfully with this patch:
        > >> * no Vim crash with the pyclewn test case
        > >> * no error reported by Valgrind with the pyclewn test case
        > >> * Vim does not crash when doing fast stepping with clewn
        > >> * netbeans does process messages when Vim is idle in normal_cmd()
        > >> * the full pyclewn regression test suite runs Ok.
        > >> [...]
        > >
        > > This will help, but I think it's still possible that Vim is updating the
        > > screen somewhere when a netbeans event is received. In functions called
        > > from normal_cmd() it's very well possible that the screen is updated
        > > directly instead of setting the must_redraw flag.
        > >
        > > One way to solve this is to set that callback_restricted flag in
        > > ui_breakcheck(). Because we don't want to handle the netbeans messages
        > > while checking if the user typed CTRL-C.
        > >
        > > It might still not be sufficient though. It's difficult to overview
        > > what happens with these asynchronous messages. And we also have to be
        > > careful not to block event handling (I think that happens when you would
        > > use the may_garbage_collect flag).
        > >
        > > Perhaps you can try out the change to ui_breakcheck() and tell us what
        > > happens. Perhaps there are other places where gui_mch_update() is
        > > called and we don't want to handle events, thus setting the flag in
        > > gui_mch_update() might be an alternative. But you can't do that
        > > always...
        >
        > Setting the callback_restricted flag in ui_breakcheck() does not work.
        > Most pyclewn automated tests fail because the netbeans messages are
        > received (and queued) in ui_breakcheck() and nb_parse_messages() never
        > gets a chance to process the received messages. This points out to
        > a problem that may occur when netbeans is run interactively: if the
        > last netbeans message is received in ui_breakcheck(), it is not
        > processed by nb_parse_messages() and appears to the user as not
        > received at all (until the next message arrives).
        > Except for that problem, this solution fixes the memory corruption as
        > expected in all the tests that do run.
        >
        > Another way to fix this is to prevent the netbeans callback to call
        > any vim function at all. This cancels the need of a global
        > callback_restricted flag. The callback queues the received messages
        > and exit the gui event loop. The messages are processed in the idle
        > loop where vim is waiting for character input.
        >
        > The following patch (also attached to this mail) implements this
        > solution for FEAT_GUI_GTK. All the tests are successful (see the list
        > of tests above, in the original post) including the 51 pyclewn tests.

        This makes sense. I'll put this in the todo list for having a closer
        look.

        > I think it is possible to implement the same solution for
        > FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
        > don't know what application can be used to test the patch (maybe Agide ?).

        This should not be changed without proper testing. So let's leave it
        as-is until we can test it.

        --
        Never under any circumstances take a sleeping pill
        and a laxative on the same night.

        /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
        /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
        \\\ download, build and distribute -- http://www.A-A-P.org ///
        \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_dev" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • Xavier de Gaye
        ... The problem of Netbeans events are handled while updating the screen, causing a crash has been fixed in Vim 7.2 for gtk. The attached patch to Vim 7.2.55
        Message 3 of 6 , Nov 29, 2008
        • 0 Attachment
          On Sun, Jun 1, 2008 at 12:05 PM, Bram Moolenaar <Bram@...> wrote:
          >
          > Xavier de Gaye wrote:
          >
          >> >> Problem:
          >> >> --------
          >> >> Vim version 7.1.298 on linux.
          >> >> Vim crashes processing received netbeans messages while redrawing the
          >> >> windows. This happens about once every five times when running the
          >> >> [...]
          >> >> The following tests have been completed successfully with this patch:
          >> >> * no Vim crash with the pyclewn test case
          >> >> * no error reported by Valgrind with the pyclewn test case
          >> >> * Vim does not crash when doing fast stepping with clewn
          >> >> * netbeans does process messages when Vim is idle in normal_cmd()
          >> >> * the full pyclewn regression test suite runs Ok.
          >> >> [...]
          >> >
          >> > This will help, but I think it's still possible that Vim is updating the
          >> > screen somewhere when a netbeans event is received. In functions called
          >> > from normal_cmd() it's very well possible that the screen is updated
          >> > directly instead of setting the must_redraw flag.
          >> >
          >> > One way to solve this is to set that callback_restricted flag in
          >> > ui_breakcheck(). Because we don't want to handle the netbeans messages
          >> > while checking if the user typed CTRL-C.
          >> >
          >> > It might still not be sufficient though. It's difficult to overview
          >> > what happens with these asynchronous messages. And we also have to be
          >> > careful not to block event handling (I think that happens when you would
          >> > use the may_garbage_collect flag).
          >> >
          >> > Perhaps you can try out the change to ui_breakcheck() and tell us what
          >> > happens. Perhaps there are other places where gui_mch_update() is
          >> > called and we don't want to handle events, thus setting the flag in
          >> > gui_mch_update() might be an alternative. But you can't do that
          >> > always...
          >>
          >> Setting the callback_restricted flag in ui_breakcheck() does not work.
          >> Most pyclewn automated tests fail because the netbeans messages are
          >> received (and queued) in ui_breakcheck() and nb_parse_messages() never
          >> gets a chance to process the received messages. This points out to
          >> a problem that may occur when netbeans is run interactively: if the
          >> last netbeans message is received in ui_breakcheck(), it is not
          >> processed by nb_parse_messages() and appears to the user as not
          >> received at all (until the next message arrives).
          >> Except for that problem, this solution fixes the memory corruption as
          >> expected in all the tests that do run.
          >>
          >> Another way to fix this is to prevent the netbeans callback to call
          >> any vim function at all. This cancels the need of a global
          >> callback_restricted flag. The callback queues the received messages
          >> and exit the gui event loop. The messages are processed in the idle
          >> loop where vim is waiting for character input.
          >>
          >> The following patch (also attached to this mail) implements this
          >> solution for FEAT_GUI_GTK. All the tests are successful (see the list
          >> of tests above, in the original post) including the 51 pyclewn tests.
          >
          > This makes sense. I'll put this in the todo list for having a closer
          > look.
          >
          >> I think it is possible to implement the same solution for
          >> FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
          >> don't know what application can be used to test the patch (maybe Agide ?).
          >
          > This should not be changed without proper testing. So let's leave it
          > as-is until we can test it.


          The problem of "Netbeans events are handled while updating the screen,
          causing a crash" has been fixed in Vim 7.2 for gtk. The attached patch
          to Vim 7.2.55 (subversion revision 1273) fixes this same problem for
          Windows.

          The following tests have been completed successfully with this patch
          and pyclewn version 0.6 on Windows:
          * Vim processes netbeans messages when idle
          * the full pyclewn regression test suite runs Ok

          Note that the crash could not be reproduced with Vim 7.2 and pyclewn,
          as the current implementation of pyclewn on Windows is too slow.

          Xavier

          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_dev" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Bram Moolenaar
          ... Thanks for the patch. Looks OK to me, although I can t try it out. I ll include it. -- This is the polymorph virus! Follow these instructions carefully:
          Message 4 of 6 , Nov 30, 2008
          • 0 Attachment
            Xavier de Gaye wrote:

            > >> >> Problem:
            > >> >> --------
            > >> >> Vim version 7.1.298 on linux.
            > >> >> Vim crashes processing received netbeans messages while redrawing the
            > >> >> windows. This happens about once every five times when running the
            > >> >> [...]
            > >> >> The following tests have been completed successfully with this patch:
            > >> >> * no Vim crash with the pyclewn test case
            > >> >> * no error reported by Valgrind with the pyclewn test case
            > >> >> * Vim does not crash when doing fast stepping with clewn
            > >> >> * netbeans does process messages when Vim is idle in normal_cmd()
            > >> >> * the full pyclewn regression test suite runs Ok.
            > >> >> [...]
            > >> >
            > >> > This will help, but I think it's still possible that Vim is updating the
            > >> > screen somewhere when a netbeans event is received. In functions called
            > >> > from normal_cmd() it's very well possible that the screen is updated
            > >> > directly instead of setting the must_redraw flag.
            > >> >
            > >> > One way to solve this is to set that callback_restricted flag in
            > >> > ui_breakcheck(). Because we don't want to handle the netbeans messages
            > >> > while checking if the user typed CTRL-C.
            > >> >
            > >> > It might still not be sufficient though. It's difficult to overview
            > >> > what happens with these asynchronous messages. And we also have to be
            > >> > careful not to block event handling (I think that happens when you would
            > >> > use the may_garbage_collect flag).
            > >> >
            > >> > Perhaps you can try out the change to ui_breakcheck() and tell us what
            > >> > happens. Perhaps there are other places where gui_mch_update() is
            > >> > called and we don't want to handle events, thus setting the flag in
            > >> > gui_mch_update() might be an alternative. But you can't do that
            > >> > always...
            > >>
            > >> Setting the callback_restricted flag in ui_breakcheck() does not work.
            > >> Most pyclewn automated tests fail because the netbeans messages are
            > >> received (and queued) in ui_breakcheck() and nb_parse_messages() never
            > >> gets a chance to process the received messages. This points out to
            > >> a problem that may occur when netbeans is run interactively: if the
            > >> last netbeans message is received in ui_breakcheck(), it is not
            > >> processed by nb_parse_messages() and appears to the user as not
            > >> received at all (until the next message arrives).
            > >> Except for that problem, this solution fixes the memory corruption as
            > >> expected in all the tests that do run.
            > >>
            > >> Another way to fix this is to prevent the netbeans callback to call
            > >> any vim function at all. This cancels the need of a global
            > >> callback_restricted flag. The callback queues the received messages
            > >> and exit the gui event loop. The messages are processed in the idle
            > >> loop where vim is waiting for character input.
            > >>
            > >> The following patch (also attached to this mail) implements this
            > >> solution for FEAT_GUI_GTK. All the tests are successful (see the list
            > >> of tests above, in the original post) including the 51 pyclewn tests.
            > >
            > > This makes sense. I'll put this in the todo list for having a closer
            > > look.
            > >
            > >> I think it is possible to implement the same solution for
            > >> FEAT_GUI_W32, but Windows support in pyclewn is not ready yet and I
            > >> don't know what application can be used to test the patch (maybe Agide ?).
            > >
            > > This should not be changed without proper testing. So let's leave it
            > > as-is until we can test it.
            >
            >
            > The problem of "Netbeans events are handled while updating the screen,
            > causing a crash" has been fixed in Vim 7.2 for gtk. The attached patch
            > to Vim 7.2.55 (subversion revision 1273) fixes this same problem for
            > Windows.
            >
            > The following tests have been completed successfully with this patch
            > and pyclewn version 0.6 on Windows:
            > * Vim processes netbeans messages when idle
            > * the full pyclewn regression test suite runs Ok
            >
            > Note that the crash could not be reproduced with Vim 7.2 and pyclewn,
            > as the current implementation of pyclewn on Windows is too slow.

            Thanks for the patch. Looks OK to me, although I can't try it out.
            I'll include it.

            --
            This is the polymorph virus! Follow these instructions carefully:
            1. Send this message to everybody you know.
            2. Format your harddisk.
            Thank you for your cooperation in spreading the most powerful virus ever!

            /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
            /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
            \\\ download, build and distribute -- http://www.A-A-P.org ///
            \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_dev" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          Your message has been successfully submitted and would be delivered to recipients shortly.