Loading ...
Sorry, an error occurred while loading the content.

Re: Spell correction ignores certain characters

Expand Messages
  • Bram Moolenaar
    ... I see it too. Also the ß in Georgenkirchstraße is not highlighted as a spell error. I wonder if this is a problem in the spell dictionary or in the
    Message 1 of 10 , May 2 6:35 AM
    • 0 Attachment
      Axel Bender wrote:

      > When using the spell correction feature ("set spelllang=de_de spell")
      > a word containing a german "ß" (0xDF) is displayed incorrectly if it
      > is found to be misspelled:
      >
      > Wir wohnen nicht in der >>>Georgenkirchstraße<<<, sondern in der
      > >>>Hauptstraße<<<.
      >
      > In this sample "Georgenkirchstraße" is flagged as misspelled (as
      > opposed to "Hauptstraße") - which would be ok - but the "ß" in the
      > flagged word is not considered as a valid word character; hence the
      > word cannot be directly added (w/o creating a selection) to the
      > exception list with e.g. "zg".

      I see it too. Also the ß in "Georgenkirchstraße" is not highlighted as
      a spell error. I wonder if this is a problem in the spell dictionary or
      in the Vim code.

      --
      Biting someone with your natural teeth is "simple assault," while biting
      someone with your false teeth is "aggravated assault."
      [real standing law in Louisana, United States of America]

      /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
      /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
      \\\ an exciting new programming language -- http://www.Zimbu.org ///
      \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • Dominique Pellé
      ... ...snip... I also see the German eszett not being highlighted by the spelling checker (bug). However, I don t see any crash using Linux x86 or Linux x86_64
      Message 2 of 10 , May 2 12:27 PM
      • 0 Attachment
        Christian Brabandt <cblists@...> wrote:

        > Hi
        >
        > On Di, 01 Mai 2012, Axel wrote:
        >
        >> When using the spell correction feature ("set spelllang=de_de spell") a word containing a german "ß" (0xDF) is displayed incorrectly if it is found to be misspelled:
        >>
        >> Wir wohnen nicht in der >>>Georgenkirchstraße<<<, sondern in der >>>Hauptstraße<<<.
        >>
        >> In this sample "Georgenkirchstraße" is flagged as misspelled (as opposed to "Hauptstraße") - which would be ok - but the "ß" in the flagged word is not considered as a valid word character; hence the word cannot be directly added (w/o creating a selection) to the exception list with e.g. "zg".
        >>
        >
        > While I can't say anything about it, I noticed that this crashes my vim
        > with a slight variation of the above sentence:
        >
        > gdb --args ./vim -u NONE -U NONE -N -c ":put ='Wir wohnen nicht in der >>>Georgenkirch straße<<<, sondern in der >>>Hauptstraße<<<'" -c 'set spell spelllang=de
        >
        > [Vim starts and puts that sentence in the buffer]
        > :norm! fsz=
        >
        >  SIGSEGV, Segmentation fault.

        ...snip...

        I also see the German eszett not being highlighted by the
        spelling checker (bug). However, I don't see any crash using
        Linux x86 or Linux x86_64 with Vim-7.3.515. I used the latest
        German dictionary which Vim downloaded automatically
        when setting ':set spelllang=de'.

        -- Dominique

        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • Christian Brabandt
        Hi Dominique! ... I guess, *smp got invalid somehow. This patch prevents, that it gets overwritten: diff --git a/src/spell.c b/src/spell.c ... +++
        Message 3 of 10 , May 9 5:21 AM
        • 0 Attachment
          Hi Dominique!

          On Mi, 02 Mai 2012, Dominique Pellé wrote:

          > Christian Brabandt <cblists@...> wrote:
          >
          > > Hi
          > >
          > > On Di, 01 Mai 2012, Axel wrote:
          > >
          > >> When using the spell correction feature ("set spelllang=de_de spell") a word containing a german "ß" (0xDF) is displayed incorrectly if it is found to be misspelled:
          > >>
          > >> Wir wohnen nicht in der >>>Georgenkirchstraße<<<, sondern in der >>>Hauptstraße<<<.
          > >>
          > >> In this sample "Georgenkirchstraße" is flagged as misspelled (as opposed to "Hauptstraße") - which would be ok - but the "ß" in the flagged word is not considered as a valid word character; hence the word cannot be directly added (w/o creating a selection) to the exception list with e.g. "zg".
          > >>
          > >
          > > While I can't say anything about it, I noticed that this crashes my vim
          > > with a slight variation of the above sentence:
          > >
          > > gdb --args ./vim -u NONE -U NONE -N -c ":put ='Wir wohnen nicht in der >>>Georgenkirch straße<<<, sondern in der >>>Hauptstraße<<<'" -c 'set spell spelllang=de
          > >
          > > [Vim starts and puts that sentence in the buffer]
          > > :norm! fsz=
          > >
          > >  SIGSEGV, Segmentation fault.
          >
          > ...snip...
          >
          > I also see the German eszett not being highlighted by the
          > spelling checker (bug). However, I don't see any crash using
          > Linux x86 or Linux x86_64 with Vim-7.3.515. I used the latest
          > German dictionary which Vim downloaded automatically
          > when setting ':set spelllang=de'.

          I guess, *smp got invalid somehow. This patch prevents, that it gets
          overwritten:

          diff --git a/src/spell.c b/src/spell.c
          --- a/src/spell.c
          +++ b/src/spell.c
          @@ -14766,6 +14766,9 @@
          z = 0;
          k = 0;
          }
          + /* prevent overwriting of *smp pointer */
          + if (smp != slang->sl_sal.ga_data)
          + smp = (salitem_T *)slang->sl_sal.ga_data;
          }


          Mit freundlichen Grüßen
          Christian
          --

          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        • Axel
          Will this get fixed in some official patch? -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are
          Message 4 of 10 , May 18 7:51 AM
          • 0 Attachment
            Will this get fixed in some official patch?

            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
          • Bram Moolenaar
            ... This patch doesn t make sense. It would suggest slang- sl_sal.ga_data is changed, or that smp is changed. I don t see how that could happen. Could this
            Message 5 of 10 , May 18 8:43 AM
            • 0 Attachment
              Christian Brabandt wrote:

              > > >> When using the spell correction feature ("set spelllang=de_de spell") a word containing a german "ß" (0xDF) is displayed incorrectly if it is found to be misspelled:
              > > >>
              > > >> Wir wohnen nicht in der >>>Georgenkirchstraße<<<, sondern in der >>>Hauptstraße<<<.
              > > >>
              > > >> In this sample "Georgenkirchstraße" is flagged as misspelled (as opposed to "Hauptstraße") - which would be ok - but the "ß" in the flagged word is not considered as a valid word character; hence the word cannot be directly added (w/o creating a selection) to the exception list with e.g. "zg".
              > > >>
              > > >
              > > > While I can't say anything about it, I noticed that this crashes my vim
              > > > with a slight variation of the above sentence:
              > > >
              > > > gdb --args ./vim -u NONE -U NONE -N -c ":put ='Wir wohnen nicht in der >>>Georgenkirch straße<<<, sondern in der >>>Hauptstraße<<<'" -c 'set spell spelllang=de
              > > >
              > > > [Vim starts and puts that sentence in the buffer]
              > > > :norm! fsz=
              > > >
              > > >  SIGSEGV, Segmentation fault.
              > >
              > > ...snip...
              > >
              > > I also see the German eszett not being highlighted by the
              > > spelling checker (bug). However, I don't see any crash using
              > > Linux x86 or Linux x86_64 with Vim-7.3.515. I used the latest
              > > German dictionary which Vim downloaded automatically
              > > when setting ':set spelllang=de'.
              >
              > I guess, *smp got invalid somehow. This patch prevents, that it gets
              > overwritten:
              >
              > diff --git a/src/spell.c b/src/spell.c
              > --- a/src/spell.c
              > +++ b/src/spell.c
              > @@ -14766,6 +14766,9 @@
              > z = 0;
              > k = 0;
              > }
              > + /* prevent overwriting of *smp pointer */
              > + if (smp != slang->sl_sal.ga_data)
              > + smp = (salitem_T *)slang->sl_sal.ga_data;
              > }

              This patch doesn't make sense. It would suggest slang->sl_sal.ga_data
              is changed, or that smp is changed. I don't see how that could happen.
              Could this be a memory access error? I could not reproduce it though.

              When using valgrind I do find this:

              ==22362== Conditional jump or move depends on uninitialised value(s)
              ==22362== at 0x81BD866: spell_soundfold_wsal (spell.c:14741)
              ==22362== by 0x81BC11B: spell_soundfold (spell.c:14106)
              ==22362== by 0x81BA588: stp_sal_score (spell.c:13189)

              Since word[] is just after smp and on the stack order is reversed, I
              suspect the length of the mch_memmove is wrong.

              --
              ARTHUR: I've said I'm sorry about the old woman, but from the behind you
              looked ...
              DENNIS: What I object to is that you automatically treat me like an inferior...
              ARTHUR: Well ... I AM king.
              "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

              /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
              /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
              \\\ an exciting new programming language -- http://www.Zimbu.org ///
              \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php
            • Dominique Pellé
              ... I m not able to reproduce this Valgrind error. Nor can I reproduce the issue in this thread. For me, smp is always equal to slang- sl_sal.ga_data. Perhaps
              Message 6 of 10 , May 18 8:50 AM
              • 0 Attachment
                Bram Moolenaar <Bram@...> wrote:

                > Christian Brabandt wrote:
                >
                >> > >> When using the spell correction feature ("set spelllang=de_de spell") a word containing a german "ß" (0xDF) is displayed incorrectly if it is found to be misspelled:
                >> > >>
                >> > >> Wir wohnen nicht in der >>>Georgenkirchstraße<<<, sondern in der >>>Hauptstraße<<<.
                >> > >>
                >> > >> In this sample "Georgenkirchstraße" is flagged as misspelled (as opposed to "Hauptstraße") - which would be ok - but the "ß" in the flagged word is not considered as a valid word character; hence the word cannot be directly added (w/o creating a selection) to the exception list with e.g. "zg".
                >> > >>
                >> > >
                >> > > While I can't say anything about it, I noticed that this crashes my vim
                >> > > with a slight variation of the above sentence:
                >> > >
                >> > > gdb --args ./vim -u NONE -U NONE -N -c ":put ='Wir wohnen nicht in der >>>Georgenkirch straße<<<, sondern in der >>>Hauptstraße<<<'" -c 'set spell spelllang=de
                >> > >
                >> > > [Vim starts and puts that sentence in the buffer]
                >> > > :norm! fsz=
                >> > >
                >> > >  SIGSEGV, Segmentation fault.
                >> >
                >> > ...snip...
                >> >
                >> > I also see the German eszett not being highlighted by the
                >> > spelling checker (bug). However, I don't see any crash using
                >> > Linux x86 or Linux x86_64 with Vim-7.3.515.  I used the latest
                >> > German dictionary which Vim downloaded automatically
                >> > when setting ':set spelllang=de'.
                >>
                >> I guess, *smp got invalid somehow. This patch prevents, that it gets
                >> overwritten:
                >>
                >> diff --git a/src/spell.c b/src/spell.c
                >> --- a/src/spell.c
                >> +++ b/src/spell.c
                >> @@ -14766,6 +14766,9 @@
                >>             z = 0;
                >>             k = 0;
                >>         }
                >> +       /* prevent overwriting of *smp pointer */
                >> +       if (smp != slang->sl_sal.ga_data)
                >> +           smp = (salitem_T *)slang->sl_sal.ga_data;
                >>      }
                >
                > This patch doesn't make sense.  It would suggest slang->sl_sal.ga_data
                > is changed, or that smp is changed.  I don't see how that could happen.
                > Could this be a memory access error?  I could not reproduce it though.
                >
                > When using valgrind I do find this:
                >
                > ==22362== Conditional jump or move depends on uninitialised value(s)
                > ==22362==    at 0x81BD866: spell_soundfold_wsal (spell.c:14741)
                > ==22362==    by 0x81BC11B: spell_soundfold (spell.c:14106)
                > ==22362==    by 0x81BA588: stp_sal_score (spell.c:13189)
                >
                > Since word[] is just after smp and on the stack order is reversed, I
                > suspect the length of the mch_memmove is wrong.

                I'm not able to reproduce this Valgrind error. Nor can I reproduce
                the issue in this thread. For me, smp is always equal to slang->sl_sal.ga_data.

                Perhaps the bug happens only with some locale? What is your locale?

                When valgrind reports "uninitialized value(s)", it can help to add
                the valgrind command line option --track-origins=yes. It gives better
                error reporting at the cost of running slightly slower (which is why
                it's off by default).

                Regards
                -- Dominique

                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php
              • Bram Moolenaar
                ... I located the problem: using strlen() on an int pointer. Not a good idea! :-) This doesn t solve the highlighting, just the potential crash. -- DENNIS:
                Message 7 of 10 , May 18 9:09 AM
                • 0 Attachment
                  Dominique Pelle wrote:

                  > Bram Moolenaar <Bram@...> wrote:
                  >
                  > > Christian Brabandt wrote:
                  > >
                  > >> > >> When using the spell correction feature ("set spelllang=de_de spell") a word containing a german "ß" (0xDF) is displayed incorrectly if it is found to be misspelled:
                  > >> > >>
                  > >> > >> Wir wohnen nicht in der >>>Georgenkirchstraße<<<, sondern in der >>>Hauptstraße<<<.
                  > >> > >>
                  > >> > >> In this sample "Georgenkirchstraße" is flagged as misspelled (as opposed to "Hauptstraße") - which would be ok - but the "ß" in the flagged word is not considered as a valid word character; hence the word cannot be directly added (w/o creating a selection) to the exception list with e.g. "zg".
                  > >> > >>
                  > >> > >
                  > >> > > While I can't say anything about it, I noticed that this crashes my vim
                  > >> > > with a slight variation of the above sentence:
                  > >> > >
                  > >> > > gdb --args ./vim -u NONE -U NONE -N -c ":put ='Wir wohnen nicht in der >>>Georgenkirch straße<<<, sondern in der >>>Hauptstraße<<<'" -c 'set spell spelllang=de
                  > >> > >
                  > >> > > [Vim starts and puts that sentence in the buffer]
                  > >> > > :norm! fsz=
                  > >> > >
                  > >> > >  SIGSEGV, Segmentation fault.
                  > >> >
                  > >> > ...snip...
                  > >> >
                  > >> > I also see the German eszett not being highlighted by the
                  > >> > spelling checker (bug). However, I don't see any crash using
                  > >> > Linux x86 or Linux x86_64 with Vim-7.3.515.  I used the latest
                  > >> > German dictionary which Vim downloaded automatically
                  > >> > when setting ':set spelllang=de'.
                  > >>
                  > >> I guess, *smp got invalid somehow. This patch prevents, that it gets
                  > >> overwritten:
                  > >>
                  > >> diff --git a/src/spell.c b/src/spell.c
                  > >> --- a/src/spell.c
                  > >> +++ b/src/spell.c
                  > >> @@ -14766,6 +14766,9 @@
                  > >>             z = 0;
                  > >>             k = 0;
                  > >>         }
                  > >> +       /* prevent overwriting of *smp pointer */
                  > >> +       if (smp != slang->sl_sal.ga_data)
                  > >> +           smp = (salitem_T *)slang->sl_sal.ga_data;
                  > >>      }
                  > >
                  > > This patch doesn't make sense.  It would suggest slang->sl_sal.ga_data
                  > > is changed, or that smp is changed.  I don't see how that could happen.
                  > > Could this be a memory access error?  I could not reproduce it though.
                  > >
                  > > When using valgrind I do find this:
                  > >
                  > > ==22362== Conditional jump or move depends on uninitialised value(s)
                  > > ==22362==    at 0x81BD866: spell_soundfold_wsal (spell.c:14741)
                  > > ==22362==    by 0x81BC11B: spell_soundfold (spell.c:14106)
                  > > ==22362==    by 0x81BA588: stp_sal_score (spell.c:13189)
                  > >
                  > > Since word[] is just after smp and on the stack order is reversed, I
                  > > suspect the length of the mch_memmove is wrong.
                  >
                  > I'm not able to reproduce this Valgrind error. Nor can I reproduce
                  > the issue in this thread. For me, smp is always equal to slang->sl_sal.ga_data.
                  >
                  > Perhaps the bug happens only with some locale? What is your locale?
                  >
                  > When valgrind reports "uninitialized value(s)", it can help to add
                  > the valgrind command line option --track-origins=yes. It gives better
                  > error reporting at the cost of running slightly slower (which is why
                  > it's off by default).

                  I located the problem: using strlen() on an int pointer. Not a good
                  idea! :-)

                  This doesn't solve the highlighting, just the potential crash.

                  --
                  DENNIS: Oh, very nice. King, eh! I expect you've got a palace and fine
                  clothes and courtiers and plenty of food. And how d'you get that? By
                  exploiting the workers! By hanging on to outdated imperialist dogma
                  which perpetuates the social and economic differences in our society!
                  "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

                  /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                  /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                  \\\ an exciting new programming language -- http://www.Zimbu.org ///
                  \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php
                • Axel Bender
                  I d like to renew my request to solve this bug... -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text
                  Message 8 of 10 , Mar 1
                  • 0 Attachment
                    I'd like to renew my request to solve this bug...

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/d/optout.
                  Your message has been successfully submitted and would be delivered to recipients shortly.