Loading ...
Sorry, an error occurred while loading the content.

Pasting some unicode characters sometimes crashes vim

Expand Messages
  • ngollan
    When pasting a high unicode character ( uFFFF) into vim, the program sometimes crashes from a SIGFPE. I m running vim from Debian testing on amd64. =====
    Message 1 of 3 , Dec 6, 2011
    • 0 Attachment
      When pasting a high unicode character (>uFFFF) into vim, the program
      sometimes crashes from a SIGFPE.

      I'm running vim from Debian testing on amd64.

      ===== Example character:
      http://www.fileformat.info/info/unicode/char/1f638/index.htm

      ===== Console output:
      Vim: Caught deadly signal FPE
      Vim: preserving files...
      Vim: Finished.
      Floating point exception

      ===== gdb output:
      Program received signal SIGFPE, Arithmetic exception.
      0x00000000004f500e in utf_convert (a=128568, table=<optimized out>,
      tableSize=<optimized out>) at mbyte.c:2780
      2780 && (a - table[start].rangeStart) %
      table[start].step == 0)
      (gdb) bt
      #0 0x00000000004f500e in utf_convert (a=128568, table=<optimized
      out>, tableSize=<optimized out>) at mbyte.c:2780
      #1 0x000000000044780d in str_foldcase (str=<optimized out>,
      orglen=<optimized out>, buf=0x7fffe9fcfaf0 "\360\237\230\270",
      buflen=81) at charset.c:467
      #2 0x000000000057689f in check_keyword_id (ccharp=<synthetic
      pointer>, cur_si=0x0, next_listp=<synthetic pointer>,
      flagsp=0x7fffe9fcfae0, endcolp=<synthetic pointer>,
      startcol=0, line=<optimized out>) at syntax.c:3305
      #3 syn_current_attr (syncing=<optimized out>, displaying=1,
      can_spell=0x0, keep_state=0) at syntax.c:1967
      #4 0x0000000000576d28 in get_syntax_attr (col=3, can_spell=0x0,
      keep_state=0) at syntax.c:1830
      #5 0x00000000005424c2 in win_line (wp=0x1462f60, lnum=15,
      startrow=14, endrow=64, nochange=0) at screen.c:4075
      #6 0x00000000005454e7 in win_update (wp=0x1462f60) at screen.c:1850
      #7 0x0000000000547bf8 in update_screen (type=<optimized out>) at
      screen.c:531
      #8 0x000000000044f1c0 in ins_redraw (ready=1) at edit.c:1549
      #9 ins_redraw (ready=1) at edit.c:1512
      #10 0x000000000045843c in edit (cmdchar=105, startln=<optimized out>,
      count=1) at edit.c:706
      #11 0x00000000004fadbc in invoke_edit (repl=<optimized out>,
      cmd=<optimized out>, startln=<optimized out>, cap=<optimized out>) at
      normal.c:9092
      #12 0x0000000000503476 in normal_cmd (oap=0x7fffe9fd02d0, toplevel=1)
      at normal.c:1193
      #13 0x00000000005bec8d in main_loop (cmdwin=0, noexmode=0) at main.c:
      1266
      #14 0x000000000043c314 in main (argc=<optimized out>, argv=<optimized
      out>) at main.c:967


      ===== :version
      VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Nov 16 2011
      00:59:56)
      Included patches:
      1-346
      Modified by pkg-vim-
      maintainers@...
      Compiled by
      buildd@...
      Huge version with GTK2 GUI. Features included (+) or not
      (-):
      +arabic +autocmd +balloon_eval +browse ++builtin_terms +byte_offset
      +cindent +clientserver +clipboard +cmdline_compl +cmdline_hist
      +cmdline_info +comments +conceal +cryptv +cscope
      +cursorbind +cursorshape +dialog_con_gui +diff +digraphs +dnd -ebcdic
      +emacs_tags +eval +ex_extra +extra_search +farsi +file_in_path
      +find_in_path +float +folding -footer +fork()
      +gettext -hangul_input +iconv +insert_expand +jumplist +keymap
      +langmap +libcall +linebreak +lispindent +listcmds +localmap +lua
      +menu +mksession +modify_fname +mouse +mouseshape
      +mouse_dec +mouse_gpm -mouse_jsbterm +mouse_netterm -mouse_sysmouse
      +mouse_xterm +mouse_urxvt +multi_byte +multi_lang -mzscheme
      +netbeans_intg +path_extra +perl +persistent_undo
      +postscript +printer +profile +python -python3 +quickfix +reltime
      +rightleft +ruby +scrollbind +signs +smartindent -sniff +startuptime
      +statusline -sun_workshop +syntax
      +tag_binary +tag_old_static -tag_any_white +tcl +terminfo
      +termresponse +textobjects +title +toolbar +user_commands +vertsplit
      +virtualedit +visual +visualextra +viminfo +vreplace
      +wildignore +wildmenu +windows +writebackup +X11 -xfontset +xim
      +xsmp_interact +xterm_clipboard -
      xterm_save
      system vimrc file: "$VIM/
      vimrc"
      user vimrc file:
      "$HOME/.vimrc"
      user exrc file:
      "$HOME/.exrc"
      system gvimrc file: "$VIM/
      gvimrc"
      user gvimrc file:
      "$HOME/.gvimrc"
      system menu file: "$VIMRUNTIME/
      menu.vim"
      fall-back for $VIM: "/usr/share/
      vim"
      Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK -
      pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/
      include -I/usr/include/atk-1.0 -I/usr/include/gd
      k-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/pixman-1 -I/usr/
      include/freetype2 -I/usr/include/libpng12 -I/usr/include/cairo -I/usr/
      include/gio-unix-2.0/ -I/usr/include/glib
      -2.0 -I/usr/lib/glib-2.0/include -g -O2 -fstack-protector --
      param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-
      security -D_FORTIFY_SOURCE=1 -I/usr/include/tc
      l8.5 -D_REENTRANT=1 -D_THREAD_SAFE=1 -
      D_LARGEFILE64_SOURCE=1
      Linking: gcc -L. -rdynamic -Wl,-export-dynamic -Wl,-E -Wl,-
      z,relro -Wl,--as-needed -o vim -pthread -lgtk-x11-2.0 -lgdk-x11-2.0 -
      latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lpangocair
      o-1.0 -lgdk_pixbuf-2.0 -lcairo -lpango-1.0 -lfreetype -lfontconfig -
      lgobject-2.0 -lgmodule-2.0 -lgthread-2.0 -lrt -lglib-2.0 -lSM -lICE -
      lXpm -lXt -lX11 -lXdmcp -lSM -lICE -lm -l
      tinfo -lnsl -lselinux -lacl -lattr -lgpm -L/usr/lib -llua5.1 -Wl,-
      E -fstack-protector -L/usr/local/lib -L/usr/lib/perl/5.14/CORE -
      lperl -ldl -lm -lpthread -lcrypt -L/usr/lib/
      python2.7/config -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -
      export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions -L/usr/lib -ltcl8.5 -
      ldl -lpthread -lieee -lm -lruby1.8 -lpthread
      -lrt -ldl -lcrypt -lm -L/usr/
      lib

      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • Dominique PellĂ©
      ... Hi I can reproduce it with Vim-7.3.364 on Linux. Just opening a file containing Unicode character 0x1f638 makes Vim crash. I see that the binary search
      Message 2 of 3 , Dec 6, 2011
      • 0 Attachment
        ngollan wrote:

        > When pasting a high unicode character (>uFFFF) into vim, the program
        > sometimes crashes from a SIGFPE.
        >
        > I'm running vim from Debian testing on amd64.
        >
        > ===== Example character:
        > http://www.fileformat.info/info/unicode/char/1f638/index.htm

        Hi

        I can reproduce it with Vim-7.3.364 on Linux. Just opening a
        file containing Unicode character 0x1f638 makes Vim crash.

        I see that the binary search fails to find a range in function utf_convert(...):

        vim/src/mbytes.c:

        2761 utf_convert(a, table, tableSize)
        2762 int a;
        2763 convertStruct table[];
        2764 int tableSize;
        2765 {
        2766 int start, mid, end; /* indices into table */
        2767
        2768 start = 0;
        2769 end = tableSize / sizeof(convertStruct);
        2770 while (start < end)
        2771 {
        2772 /* need to search further */
        2773 mid = (end + start) /2;
        2774 if (table[mid].rangeEnd < a)
        2775 start = mid + 1;
        2776 else
        2777 end = mid;
        2778 }
        2779 if (table[start].rangeStart <= a && a <= table[start].rangeEnd
        2780 && (a - table[start].rangeStart) % table[start].step == 0)
        2781 return (a + table[start].offset);
        2782 else
        2783 return a;
        2784 }

        The start, mid, end variables in utf_convert evolve as follow
        during the binary search:

        start=0 mid=78 end=156
        start=79 mid=117 end=156
        start=118 mid=137 end=156
        start=138 mid=147 end=156
        start=148 mid=152 end=156
        start=153 mid=154 end=156
        start=155 mid=155 end=156
        start=156 mid=155 end=156

        Then at line mbytes.c:2779, table[start] it used
        which accesses beyond the end of foldCase[] array.

        Interestingly, Valgrind memory checker does
        not detect this (since it's an overflow in a
        global variable) but the address-sanitizer tool
        available at...
        http://code.google.com/p/address-sanitizer/
        ... does detect it and says:

        READ of size 4 at 0x000000afc440 thread T0
        #0 0x5e63c2 in utf_convert mbyte.c:0
        #1 0x670dbf in vim_regexec_both regexp.c:0
        #2 0x67110c in vim_regexec_nl ??:0
        #3 0x4bdb86 in eval4 eval.c:0
        #4 0x4bcc94 in eval3 eval.c:0
        #5 0x4bc844 in eval2 eval.c:0
        #6 0x47b7be in eval1 eval.c:0
        #7 0x47af5e in eval0 eval.c:0
        #8 0x47aa29 in eval_to_bool ??:0
        #9 0x50f644 in ex_if ??:0
        #10 0x4efc7a in do_one_cmd ex_docmd.c:0
        #11 0x4e9383 in do_cmdline ??:0
        #12 0x4e6253 in do_source ??:0
        #13 0x4e506a in do_in_runtimepath ??:0
        #14 0x4efc7a in do_one_cmd ex_docmd.c:0
        #15 0x4e9383 in do_cmdline ??:0
        #16 0x54340e in apply_autocmds_group fileio.c:0
        #17 0x53361f in apply_autocmds_exarg fileio.c:0
        #18 0x532d94 in readfile ??:0
        #19 0x4321cc in open_buffer ??:0
        #20 0x7bc94a in create_windows main.c:0
        #21 0x7b7582 in main ??:0
        #22 0x7faffcdaed8e in __libc_start_main
        /build/buildd/eglibc-2.12.1/csu/libc-start.c:258
        #23 0x431e19 in _start ??:0
        0x000000afc440 is located 0 bytes to the right of global variable
        'foldCase' (0xafba80) of size 2496

        Attached patch fixes the crash.

        Regards
        -- Dominique

        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • Bram Moolenaar
        ... Thank you very much for the analysis and the patch. I m very surprised this was not detected earlier. -- A man is incomplete until he s married ... and
        Message 3 of 3 , Dec 6, 2011
        • 0 Attachment
          Dominique Pelle wrote:

          > ngollan wrote:
          >
          > > When pasting a high unicode character (>uFFFF) into vim, the program
          > > sometimes crashes from a SIGFPE.
          > >
          > > I'm running vim from Debian testing on amd64.
          > >
          > > ===== Example character:
          > > http://www.fileformat.info/info/unicode/char/1f638/index.htm
          >
          > Hi
          >
          > I can reproduce it with Vim-7.3.364 on Linux. Just opening a
          > file containing Unicode character 0x1f638 makes Vim crash.
          >
          > I see that the binary search fails to find a range in function utf_convert(...):
          >
          > vim/src/mbytes.c:
          >
          > 2761 utf_convert(a, table, tableSize)
          > 2762 int a;
          > 2763 convertStruct table[];
          > 2764 int tableSize;
          > 2765 {
          > 2766 int start, mid, end; /* indices into table */
          > 2767
          > 2768 start = 0;
          > 2769 end = tableSize / sizeof(convertStruct);
          > 2770 while (start < end)
          > 2771 {
          > 2772 /* need to search further */
          > 2773 mid = (end + start) /2;
          > 2774 if (table[mid].rangeEnd < a)
          > 2775 start = mid + 1;
          > 2776 else
          > 2777 end = mid;
          > 2778 }
          > 2779 if (table[start].rangeStart <= a && a <= table[start].rangeEnd
          > 2780 && (a - table[start].rangeStart) % table[start].step == 0)
          > 2781 return (a + table[start].offset);
          > 2782 else
          > 2783 return a;
          > 2784 }
          >
          > The start, mid, end variables in utf_convert evolve as follow
          > during the binary search:
          >
          > start=0 mid=78 end=156
          > start=79 mid=117 end=156
          > start=118 mid=137 end=156
          > start=138 mid=147 end=156
          > start=148 mid=152 end=156
          > start=153 mid=154 end=156
          > start=155 mid=155 end=156
          > start=156 mid=155 end=156
          >
          > Then at line mbytes.c:2779, table[start] it used
          > which accesses beyond the end of foldCase[] array.
          >
          > Interestingly, Valgrind memory checker does
          > not detect this (since it's an overflow in a
          > global variable) but the address-sanitizer tool
          > available at...
          > http://code.google.com/p/address-sanitizer/
          > ... does detect it and says:
          >
          > READ of size 4 at 0x000000afc440 thread T0
          > #0 0x5e63c2 in utf_convert mbyte.c:0
          > #1 0x670dbf in vim_regexec_both regexp.c:0
          > #2 0x67110c in vim_regexec_nl ??:0
          > #3 0x4bdb86 in eval4 eval.c:0
          > #4 0x4bcc94 in eval3 eval.c:0
          > #5 0x4bc844 in eval2 eval.c:0
          > #6 0x47b7be in eval1 eval.c:0
          > #7 0x47af5e in eval0 eval.c:0
          > #8 0x47aa29 in eval_to_bool ??:0
          > #9 0x50f644 in ex_if ??:0
          > #10 0x4efc7a in do_one_cmd ex_docmd.c:0
          > #11 0x4e9383 in do_cmdline ??:0
          > #12 0x4e6253 in do_source ??:0
          > #13 0x4e506a in do_in_runtimepath ??:0
          > #14 0x4efc7a in do_one_cmd ex_docmd.c:0
          > #15 0x4e9383 in do_cmdline ??:0
          > #16 0x54340e in apply_autocmds_group fileio.c:0
          > #17 0x53361f in apply_autocmds_exarg fileio.c:0
          > #18 0x532d94 in readfile ??:0
          > #19 0x4321cc in open_buffer ??:0
          > #20 0x7bc94a in create_windows main.c:0
          > #21 0x7b7582 in main ??:0
          > #22 0x7faffcdaed8e in __libc_start_main
          > /build/buildd/eglibc-2.12.1/csu/libc-start.c:258
          > #23 0x431e19 in _start ??:0
          > 0x000000afc440 is located 0 bytes to the right of global variable
          > 'foldCase' (0xafba80) of size 2496
          >
          > Attached patch fixes the crash.

          Thank you very much for the analysis and the patch.
          I'm very surprised this was not detected earlier.

          --
          A man is incomplete until he's married ... and then he's finished!

          /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
          /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
          \\\ an exciting new programming language -- http://www.Zimbu.org ///
          \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        Your message has been successfully submitted and would be delivered to recipients shortly.