Loading ...
Sorry, an error occurred while loading the content.

Bug: Misinterpretation of key sequence

Expand Messages
  • Dominic Humphries
    Entering the commands to leave insert mode & insert a new line above the current one, beginning with certain letters, results in punctuation on the current
    Message 1 of 8 , Apr 14, 2014
    • 0 Attachment
      Entering the commands to leave insert mode & insert a new line above the current one, beginning with certain letters, results in punctuation on the current line instead.

      To reproduce:
      Start vim, *quickly* type the keys: i<Esc>Om

      Expected result:
      New line created above the current one with the letter 'm' in it

      Actual result:
      Hyphen inserted on current line

      Also affects:
      Letters j,k,n,o
      j gives *, k gives +, m gives -, n gives ., o gives /

      Vim version: 7.4 (as installed by Ubuntu 13.10)
      Happens on my machine, my co-worker's machine, several VMs and to some (not all) of the people I asked about it on IRC. Definitely requires that the typing is done quickly. Causes me problems very frequently because I write a lot of Perl and when I go to declare a variable, I get "-y $foo" on the current line instead of "my $foo" on the previous line.

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/d/optout.
    • Ben Fritz
      ... This looks like you re using Vim in a terminal, and you are typing fast enough for Vim to interpret your keystrokes as terminal key codes. What are your
      Message 2 of 8 , Apr 14, 2014
      • 0 Attachment
        On Monday, April 14, 2014 5:29:33 AM UTC-5, Dominic Humphries wrote:
        > Entering the commands to leave insert mode & insert a new line above the current one, beginning with certain letters, results in punctuation on the current line instead.
        >
        > To reproduce:
        > Start vim, *quickly* type the keys: i<Esc>Om
        >
        > Expected result:
        > New line created above the current one with the letter 'm' in it
        >
        > Actual result:
        > Hyphen inserted on current line
        >
        > Also affects:
        > Letters j,k,n,o
        > j gives *, k gives +, m gives -, n gives ., o gives /
        >
        > Vim version: 7.4 (as installed by Ubuntu 13.10)
        > Happens on my machine, my co-worker's machine, several VMs and to some (not all) of the people I asked about it on IRC. Definitely requires that the typing is done quickly. Causes me problems very frequently because I write a lot of Perl and when I go to declare a variable, I get "-y $foo" on the current line instead of "my $foo" on the previous line.

        This looks like you're using Vim in a terminal, and you are typing fast enough for Vim to interpret your keystrokes as terminal key codes.

        What are your 'ttimeout' and 'ttimeoutlen' options set to? Normally you can fix problems like this by setting 'ttimeout' to ON and 'ttimeoutlen' to a small value, like 100 (ttimeoutlen is in milliseconds). See the help for both options for details.

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/d/optout.
      • Tony Mechelynck
        ... This sounds like a condition of badly configured timeouts. Try adding the following line to your vimrc: set timeout timeoutlen=5000 ttimeoutlen=100 where:
        Message 3 of 8 , Apr 14, 2014
        • 0 Attachment
          On 14/04/14 12:29, Dominic Humphries wrote:
          > Entering the commands to leave insert mode & insert a new line above the current one, beginning with certain letters, results in punctuation on the current line instead.
          >
          > To reproduce:
          > Start vim, *quickly* type the keys: i<Esc>Om
          >
          > Expected result:
          > New line created above the current one with the letter 'm' in it
          >
          > Actual result:
          > Hyphen inserted on current line
          >
          > Also affects:
          > Letters j,k,n,o
          > j gives *, k gives +, m gives -, n gives ., o gives /
          >
          > Vim version: 7.4 (as installed by Ubuntu 13.10)
          > Happens on my machine, my co-worker's machine, several VMs and to some (not all) of the people I asked about it on IRC. Definitely requires that the typing is done quickly. Causes me problems very frequently because I write a lot of Perl and when I go to declare a variable, I get "-y $foo" on the current line instead of "my $foo" on the previous line.
          >

          This sounds like a condition of badly configured timeouts. Try adding
          the following line to your vimrc:

          set timeout timeoutlen=5000 ttimeoutlen=100

          where:
          - timeouts are in milliseconds, you may vary them as follows:
          - ttimeoutlen (here one-tenth of a second) is shorter than the time
          between two keypresses at your fastest typing speed but longer than the
          time between two successive bytes received as part of the keycode for a
          single key.
          - timeoutlen (here 5 seconds) is longer than the time between two
          successive keypreses for the {lhs} of a mapping at your slowest typing
          speed, but short enough to let you avoid using the mapping by waiting
          longer than that without becoming impatient.

          Then save your vimrc, close Vim and restart it. Do you still have the
          problem after that?


          Best regards,
          Tony.
          --
          I have the simplest tastes. I am always satisfied with the best.
          -- Oscar Wilde

          --
          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_dev" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
          For more options, visit https://groups.google.com/d/optout.
        • Paul "LeoNerd" Evans
          On Mon, 14 Apr 2014 04:56:00 -0700 (PDT) ... ESC O is SS3 ... SS3 j/k/n/o are the codes XTerm and similar use for number keypad symbols. ... This /once again/
          Message 4 of 8 , Apr 14, 2014
          • 0 Attachment
            On Mon, 14 Apr 2014 04:56:00 -0700 (PDT)
            Ben Fritz <fritzophrenic@...> wrote:

            > > To reproduce:
            > > Start vim, *quickly* type the keys: i<Esc>Om

            ESC O is SS3

            > > Expected result:
            > > New line created above the current one with the letter 'm' in it
            > >
            > > Actual result:
            > > Hyphen inserted on current line
            > >
            > > Also affects:
            > > Letters j,k,n,o
            > > j gives *, k gives +, m gives -, n gives ., o gives /

            SS3 j/k/n/o are the codes XTerm and similar use for number keypad
            symbols.

            > > Vim version: 7.4 (as installed by Ubuntu 13.10)
            > > Happens on my machine, my co-worker's machine, several VMs and to
            > > some (not all) of the people I asked about it on IRC. Definitely
            > > requires that the typing is done quickly. Causes me problems very
            > > frequently because I write a lot of Perl and when I go to declare a
            > > variable, I get "-y $foo" on the current line instead of "my $foo"
            > > on the previous line.
            >
            > This looks like you're using Vim in a terminal, and you are typing
            > fast enough for Vim to interpret your keystrokes as terminal key
            > codes.

            This /once again/ is another fallout from my seemingly never-ending
            point of "can Vim please use proper keyboard input handling instead of
            stupid 1980s stuff"; whereupon an entire class of bug like this will
            just disappear.

            Specifically in this case, what vim /ought/ to do but doesn't, is set
            the terminal into "8-bit" mode, using the S8C1T sequence. After doing
            that, these keypad SS3 sequences will be sent using /only/ the SS3 byte
            (0x8f) and never by Escape O. After doing this, vim will no longer need
            to rely on stupid timing tricks (see ttimeout and friends), because any
            received Escape (0x1b) byte will be a REAL <Escape> keypress and never
            a sequence.

            Please please Bram - please look into this. If you fix this, there's a
            huge class of user-facing problems like this which will never again
            surface.

            --
            Paul "LeoNerd" Evans

            leonerd@...
            http://www.leonerd.org.uk/ | https://metacpan.org/author/PEVANS
          • Bram Moolenaar
            ... Whether the terminal is in 8 bit mode is the choice of the user (or the administrator or the system). Vim switching the terminal to 8 bit mode might break
            Message 5 of 8 , Apr 14, 2014
            • 0 Attachment
              Paul Evans wrote:

              > On Mon, 14 Apr 2014 04:56:00 -0700 (PDT)
              > Ben Fritz <fritzophrenic@...> wrote:
              >
              > > > To reproduce:
              > > > Start vim, *quickly* type the keys: i<Esc>Om
              >
              > ESC O is SS3
              >
              > > > Expected result:
              > > > New line created above the current one with the letter 'm' in it
              > > >
              > > > Actual result:
              > > > Hyphen inserted on current line
              > > >
              > > > Also affects:
              > > > Letters j,k,n,o
              > > > j gives *, k gives +, m gives -, n gives ., o gives /
              >
              > SS3 j/k/n/o are the codes XTerm and similar use for number keypad
              > symbols.
              >
              > > > Vim version: 7.4 (as installed by Ubuntu 13.10)
              > > > Happens on my machine, my co-worker's machine, several VMs and to
              > > > some (not all) of the people I asked about it on IRC. Definitely
              > > > requires that the typing is done quickly. Causes me problems very
              > > > frequently because I write a lot of Perl and when I go to declare a
              > > > variable, I get "-y $foo" on the current line instead of "my $foo"
              > > > on the previous line.
              > >
              > > This looks like you're using Vim in a terminal, and you are typing
              > > fast enough for Vim to interpret your keystrokes as terminal key
              > > codes.
              >
              > This /once again/ is another fallout from my seemingly never-ending
              > point of "can Vim please use proper keyboard input handling instead of
              > stupid 1980s stuff"; whereupon an entire class of bug like this will
              > just disappear.
              >
              > Specifically in this case, what vim /ought/ to do but doesn't, is set
              > the terminal into "8-bit" mode, using the S8C1T sequence. After doing
              > that, these keypad SS3 sequences will be sent using /only/ the SS3 byte
              > (0x8f) and never by Escape O. After doing this, vim will no longer need
              > to rely on stupid timing tricks (see ttimeout and friends), because any
              > received Escape (0x1b) byte will be a REAL <Escape> keypress and never
              > a sequence.
              >
              > Please please Bram - please look into this. If you fix this, there's a
              > huge class of user-facing problems like this which will never again
              > surface.

              Whether the terminal is in 8 bit mode is the choice of the user (or the
              administrator or the system). Vim switching the terminal to 8 bit mode
              might break lots of things. Not sure what and when, since we'll
              probably only find out after doing it. Vim should be able to detect the
              terminal being in 8 bit mode and use that.

              So it's a matter of the user switching the terminal to 8 bit mode.

              --
              Keyboard not found. Think ENTER to continue.

              /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
              /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
              \\\ an exciting new programming language -- http://www.Zimbu.org ///
              \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/d/optout.
            • Paul "LeoNerd" Evans
              On Tue, 15 Apr 2014 08:10:52 +0200 ... It s nothing to do with the user. There s a control sequence to request it. It s called S8C1T. Vim already switches the
              Message 6 of 8 , Apr 15, 2014
              • 0 Attachment
                On Tue, 15 Apr 2014 08:10:52 +0200
                Bram Moolenaar <Bram@...> wrote:

                > Whether the terminal is in 8 bit mode is the choice of the user (or
                > the administrator or the system). Vim switching the terminal to 8 bit
                > mode might break lots of things. Not sure what and when, since we'll
                > probably only find out after doing it. Vim should be able to detect
                > the terminal being in 8 bit mode and use that.
                >
                > So it's a matter of the user switching the terminal to 8 bit mode.

                It's nothing to do with the user. There's a control sequence to request
                it. It's called S8C1T. Vim already switches the terminal into alternate
                buffer, keypad-application mode, activates mouse tracking, and possibly
                a few other things as well. One more can't hurt. You just switch it
                back to 7-bit mode on shutdown, the same as vim does with switching off
                alternate buffer, setting cursor-application mode and deactivating
                mouse tracking.

                This one especially is a very safe thing to do. The logic is:

                * Always be willing to accept the one-byte CSI (0x9d) and SS3 (0x8f)
                encodings of sequences

                * Send S8C1T (Esc Sp G) on startup, then make any reporting query that
                would reply with a CSI response

                * If such a response is received in 8-bit encoding, then you know the
                terminal accepted S8C1T, and you can be sure that all keys will now
                be sent in 8-bit mode. Therefore, any received Escape (0x1b) bytes
                must be Alt- prefixes or the real Escape key, and can never be CSI
                or SS3 leaders

                If for some reason the terminal doesn't understand S8C1T then it
                doesn't matter. You'll simply never receive that 8-bit response, so
                you'll never trigger the logic in step 3 to disable the Escape-prefixed
                versions of CSI and SS3.

                If you're going to claim that switching on S8C1T is dangerous and the
                user's problem, then I'm going to claim so is setting alternate buffer,
                keypad-application and mouse tracking, and that vim should stop doing
                those as well and leave them up to the user to set.

                --
                Paul "LeoNerd" Evans

                leonerd@...
                http://www.leonerd.org.uk/ | https://metacpan.org/author/PEVANS

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/d/optout.
              • Bram Moolenaar
                ... This is a different settings with very different potential problems. That the other modes work gives no indication that 8-bit mode will work. Think of
                Message 7 of 8 , Apr 16, 2014
                • 0 Attachment
                  > On Tue, 15 Apr 2014 08:10:52 +0200
                  > Bram Moolenaar <Bram@...> wrote:
                  >
                  > > Whether the terminal is in 8 bit mode is the choice of the user (or
                  > > the administrator or the system). Vim switching the terminal to 8 bit
                  > > mode might break lots of things. Not sure what and when, since we'll
                  > > probably only find out after doing it. Vim should be able to detect
                  > > the terminal being in 8 bit mode and use that.
                  > >
                  > > So it's a matter of the user switching the terminal to 8 bit mode.
                  >
                  > It's nothing to do with the user. There's a control sequence to request
                  > it. It's called S8C1T. Vim already switches the terminal into alternate
                  > buffer, keypad-application mode, activates mouse tracking, and possibly
                  > a few other things as well. One more can't hurt. You just switch it
                  > back to 7-bit mode on shutdown, the same as vim does with switching off
                  > alternate buffer, setting cursor-application mode and deactivating
                  > mouse tracking.

                  This is a different settings with very different potential problems.
                  That the other modes work gives no indication that 8-bit mode will work.

                  Think of this: why do terminal emulators start in 7-bit mode anyway?
                  If 8-bit mode is so great, why don't they start in 8-bit mode?

                  > This one especially is a very safe thing to do. The logic is:
                  >
                  > * Always be willing to accept the one-byte CSI (0x9d) and SS3 (0x8f)
                  > encodings of sequences
                  >
                  > * Send S8C1T (Esc Sp G) on startup, then make any reporting query that
                  > would reply with a CSI response
                  >
                  > * If such a response is received in 8-bit encoding, then you know the
                  > terminal accepted S8C1T, and you can be sure that all keys will now
                  > be sent in 8-bit mode. Therefore, any received Escape (0x1b) bytes
                  > must be Alt- prefixes or the real Escape key, and can never be CSI
                  > or SS3 leaders

                  In the mean time the user may have typed something, screen has
                  refreshed, other codes may be received. All this is asynchronous, there
                  are many ways in which it can fail. Vim may even exit before 8-bit mode
                  is really working.

                  > If for some reason the terminal doesn't understand S8C1T then it
                  > doesn't matter. You'll simply never receive that 8-bit response, so
                  > you'll never trigger the logic in step 3 to disable the Escape-prefixed
                  > versions of CSI and SS3.

                  It may also happen that the terminal works, but the communication
                  channel has a problem with 8 bits (using parity bit, from the old
                  days?).

                  > If you're going to claim that switching on S8C1T is dangerous and the
                  > user's problem, then I'm going to claim so is setting alternate buffer,
                  > keypad-application and mouse tracking, and that vim should stop doing
                  > those as well and leave them up to the user to set.

                  All these are dangerous, but the ones that are being used have been
                  tested, debugged and problems have been fixed. Adding a new one means
                  it will take some time before it works reliable (or we would have to
                  revert the change if there are problems that can't be fixed).

                  Why don't you ask the maintainers of terminal emulaters to use 8-bit
                  mode by default?

                  --
                  hundred-and-one symptoms of being an internet addict:
                  58. You turn on your computer and turn off your wife.

                  /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                  /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                  \\\ an exciting new programming language -- http://www.Zimbu.org ///
                  \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php

                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/d/optout.
                • Paul "LeoNerd" Evans
                  On Wed, 16 Apr 2014 12:48:13 +0200 ... That s fine; we re not /relying/ on 8-bit mode to work. If we don t receive an 8-bit-clean CSI reply, we ll still accept
                  Message 8 of 8 , Apr 16, 2014
                  • 0 Attachment
                    On Wed, 16 Apr 2014 12:48:13 +0200
                    Bram Moolenaar <Bram@...> wrote:

                    > This is a different settings with very different potential problems.
                    > That the other modes work gives no indication that 8-bit mode will
                    > work.

                    That's fine; we're not /relying/ on 8-bit mode to work. If we don't
                    receive an 8-bit-clean CSI reply, we'll still accept the 7-bit
                    ESC-prefixed encodings anyway, and things will continue to work exactly
                    as they do now. But if we did receive it, then we can do the user a
                    favour and remove the timing-ambiguity of ESC-prefixing.

                    > Think of this: why do terminal emulators start in 7-bit mode anyway?
                    > If 8-bit mode is so great, why don't they start in 8-bit mode?

                    Because most programs are still doing keyboard sequence parsing simply
                    by trying to match the fixed database of strings that termcap/terminfo
                    gives, and those strings almost invariably are the ESC-prefixed 7-bit
                    versions.

                    If terminals defaulted to start in 8-bit mode then bash/readline would
                    not recognise these keys any more. So it starts off. For the same
                    reason that the number keypad starts in keypad-cursor mode, and mouse
                    tracking starts off disabled. readline and other programs just don't
                    recognise them.

                    But vim /does/ recognise those - vim can cope just fine with receving
                    keypad-application sequences, and mouse reporting events, so it enables
                    those things. Vim could just as easily cope with receiving 8-bit
                    CSI/SS3s.

                    > In the mean time the user may have typed something, screen has
                    > refreshed, other codes may be received. All this is asynchronous,
                    > there are many ways in which it can fail. Vim may even exit before
                    > 8-bit mode is really working.

                    And that's all fine. We'll still correctly interpret most keypresses
                    that received. We're not relying on definitely receiving it. If
                    we /happen/ to receive it, then we can help the user by removing the
                    timing wait on ESC-prefix.

                    There likely will be a short delay on startup, while that round-trip
                    happens. During that time it is exceedingly unlikely the user wanted to
                    press <Escape>, because vim starts up in normal mode anyway. They'd
                    have to have entered insert mode, typed something, then left it again
                    for this even to matter.

                    > > If for some reason the terminal doesn't understand S8C1T then it
                    > > doesn't matter. You'll simply never receive that 8-bit response, so
                    > > you'll never trigger the logic in step 3 to disable the
                    > > Escape-prefixed versions of CSI and SS3.
                    >
                    > It may also happen that the terminal works, but the communication
                    > channel has a problem with 8 bits (using parity bit, from the old
                    > days?).

                    I suppose it's possible they are using a vim from 2014 and a terminal
                    from 2014, over a serial connection from 1970. In that case, UTF-8 and
                    a whole bunch of other things will also break.

                    In the exceedingly-unlikely case that their serial line can't transport
                    8-bit sequences, they could disable this feature. As they'd almost
                    certainly know about it because many other modern things like UTF-8
                    won't work either.

                    > > If you're going to claim that switching on S8C1T is dangerous and
                    > > the user's problem, then I'm going to claim so is setting alternate
                    > > buffer, keypad-application and mouse tracking, and that vim should
                    > > stop doing those as well and leave them up to the user to set.
                    >
                    > All these are dangerous, but the ones that are being used have been
                    > tested, debugged and problems have been fixed. Adding a new one means
                    > it will take some time before it works reliable (or we would have to
                    > revert the change if there are problems that can't be fixed).

                    OK, then lets make it non-default. For people who want to be able to
                    type Escape reliably they can

                    :set use8bit

                    in their .vimrc; for anyone else they won't notice a difference. Also
                    this neatly solves the problem of "what if my 1970s-era serial line
                    can't transport 8-bit data?".

                    > Why don't you ask the maintainers of terminal emulaters to use 8-bit
                    > mode by default?

                    We can't, as you well know.

                    Seriously Bram; you and I are probably the two people on the planet
                    most aware of the fact that if terminals were to suddenly start sending
                    8-bit versions of CSI and SS3 for keypresses, then vim would not work.
                    It doesn't understand them by default. Additionally, neither would
                    bash/readline, irssi, mutt,... practically any terminal-based program,
                    that wasn't specifically designed to cope with it (i.e. by doing
                    actual parsing and not just dumb terminfo-string-prefix-eating).

                    DEC in their wisdom /knew/ this change would break older programs that
                    didn't understand it, that's why it was an optional non-default on the
                    VT2xx series, where applications can enable it if they know they can
                    cope. That was released in 1983; I think in the intervening 31 years we
                    have just about progressed the state of art in terminal applications to
                    the point they can cope with it now.

                    Don't you?

                    --
                    Paul "LeoNerd" Evans

                    leonerd@...
                    http://www.leonerd.org.uk/ | https://metacpan.org/author/PEVANS

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/d/optout.
                  Your message has been successfully submitted and would be delivered to recipients shortly.