Loading ...
Sorry, an error occurred while loading the content.

pyeval() error

Expand Messages
  • Marcin Szamotulski
    Hello, I have some issues with the pyeval() function. Let say I have a python ... Then ... Brings two errors: E859: Failed to convert returned python object
    Message 1 of 11 , Sep 1, 2012
    View Source
    • 0 Attachment
      Hello,

      I have some issues with the pyeval() function. Let say I have a python
      dictionary where the keys are integers, for example:

      :py x={1:''}

      Then
      :echo pyeval("x")

      Brings two errors:

      E859: Failed to convert returned python object to vim value
      E685: Internal error: echo_string()

      However if the keys are strings:
      :py x={'1':''}

      everything works fine.

      Best regards,
      Marcin






      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • ZyX
      ... Internal error should not happen and is a bug, attached patch that fixes this. E859 however is not a bug: vim dictionaries can have only string keys. # HG
      Message 2 of 11 , Sep 1, 2012
      View Source
      • 0 Attachment
        суббота, 1 сентября 2012 г., 16:24:54 UTC+4 пользователь Marcin Szamotulski написал:
        > I have some issues with the pyeval() function. Let say I have a python
        > dictionary where the keys are integers, for example:
        >
        > :py x={1:''}
        >
        > Then
        >
        > :echo pyeval("x")
        >
        > Brings two errors:
        >
        > E859: Failed to convert returned python object to vim value
        > E685: Internal error: echo_string()
        >
        > However if the keys are strings:
        >
        > :py x={'1':''}
        > >
        > everything works fine.

        Internal error should not happen and is a bug, attached patch that fixes this.

        E859 however is not a bug: vim dictionaries can have only string keys.

        # HG changeset patch
        # User ZyX <kp-pav@...>
        # Date 1346507519 -14400
        # Node ID 744a019963bcce46daa52f8dc61c79415c47bf45
        # Parent c0ac5ba6624303f459d0b24f5fe98b1269778213
        Fixed internal error in case it failed to convert python object to vim value

        diff -r c0ac5ba66243 -r 744a019963bc src/if_python.c
        --- a/src/if_python.c Wed Aug 29 18:50:29 2012 +0200
        +++ b/src/if_python.c Sat Sep 01 17:51:34 2012 +0400
        @@ -1787,6 +1787,10 @@
        case VAR_DICT: ++rettv->vval.v_dict->dv_refcount; break;
        case VAR_LIST: ++rettv->vval.v_list->lv_refcount; break;
        case VAR_FUNC: func_ref(rettv->vval.v_string); break;
        + case VAR_UNKNOWN:
        + rettv->v_type = VAR_NUMBER;
        + rettv->vval.v_number = 0;
        + break;
        }
        }

        diff -r c0ac5ba66243 -r 744a019963bc src/if_python3.c
        --- a/src/if_python3.c Wed Aug 29 18:50:29 2012 +0200
        +++ b/src/if_python3.c Sat Sep 01 17:51:34 2012 +0400
        @@ -1817,6 +1817,10 @@
        case VAR_DICT: ++rettv->vval.v_dict->dv_refcount; break;
        case VAR_LIST: ++rettv->vval.v_list->lv_refcount; break;
        case VAR_FUNC: func_ref(rettv->vval.v_string); break;
        + case VAR_UNKNOWN:
        + rettv->v_type = VAR_NUMBER;
        + rettv->vval.v_number = 0;
        + break;
        }
        }

        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • Marcin Szamotulski
        ... Prior to pyeval() function I was using vim.command( let dict=%s % python_dict) inside python code. I think this was a standard solution to translate
        Message 3 of 11 , Sep 1, 2012
        View Source
        • 0 Attachment
          On 07:00 Sat 01 Sep , ZyX wrote:
          > суббота, 1 сентября 2012 г., 16:24:54 UTC+4 пользователь Marcin Szamotulski написал:
          > > I have some issues with the pyeval() function. Let say I have a python
          > > dictionary where the keys are integers, for example:
          > >
          > > :py x={1:''}
          > >
          > > Then
          > >
          > > :echo pyeval("x")
          > >
          > > Brings two errors:
          > >
          > > E859: Failed to convert returned python object to vim value
          > > E685: Internal error: echo_string()
          > >
          > > However if the keys are strings:
          > >
          > > :py x={'1':''}
          > > >
          > > everything works fine.
          >
          > Internal error should not happen and is a bug, attached patch that fixes this.
          >
          > E859 however is not a bug: vim dictionaries can have only string keys.

          Prior to pyeval() function I was using
          vim.command("let dict=%s" % python_dict)
          inside python code. I think this was a standard solution to translate
          python objects like dictionaries into vim language. I think it would be
          nice if pyeval did the translation of keys into strings it self. This
          error might occur in quite a few plugins. Otherwise it is nice to add
          a comment in the documentation of pyeval().

          Best,
          Marcin

          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        • Marcin Szamotulski
          ... I looked into the documentation in the mercurial (I should check it before but my vim has bit old runtime tree) and it suggests that the keys are converted
          Message 4 of 11 , Sep 1, 2012
          View Source
          • 0 Attachment
            On 17:18 Sat 01 Sep , Marcin Szamotulski wrote:
            > On 07:00 Sat 01 Sep , ZyX wrote:
            > > суббота, 1 сентября 2012 г., 16:24:54 UTC+4 пользователь Marcin Szamotulski написал:
            > > > I have some issues with the pyeval() function. Let say I have a python
            > > > dictionary where the keys are integers, for example:
            > > >
            > > > :py x={1:''}
            > > >
            > > > Then
            > > >
            > > > :echo pyeval("x")
            > > >
            > > > Brings two errors:
            > > >
            > > > E859: Failed to convert returned python object to vim value
            > > > E685: Internal error: echo_string()
            > > >
            > > > However if the keys are strings:
            > > >
            > > > :py x={'1':''}
            > > > >
            > > > everything works fine.
            > >
            > > Internal error should not happen and is a bug, attached patch that fixes this.
            > >
            > > E859 however is not a bug: vim dictionaries can have only string keys.
            >
            > Prior to pyeval() function I was using
            > vim.command("let dict=%s" % python_dict)
            > inside python code. I think this was a standard solution to translate
            > python objects like dictionaries into vim language. I think it would be
            > nice if pyeval did the translation of keys into strings it self. This
            > error might occur in quite a few plugins. Otherwise it is nice to add
            > a comment in the documentation of pyeval().
            >
            > Best,
            > Marcin

            I looked into the documentation in the mercurial (I should check it
            before but my vim has bit old runtime tree) and it suggests that the
            keys are converted to strings:

            Dictionaries are represented as Vim |Dictionary| type with
            keys converted to strings.

            So or E859 is a bug or the documentation should be changed.

            Best,
            Marcin

            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
          • Yoshiaki Miura
            ... -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit
            Message 5 of 11 , Sep 1, 2012
            View Source
            • 0 Attachment
              Marcin Szamotulski <mszamot@...> wrote:

              >Hello,
              >
              >I have some issues with the pyeval() function. Let say I have a python
              >dictionary where the keys are integers, for example:
              >
              >:py x={1:''}
              >
              >Then
              >:echo pyeval("x")
              >
              >Brings two errors:
              >
              >E859: Failed to convert returned python object to vim value
              >E685: Internal error: echo_string()
              >
              >However if the keys are strings:
              >:py x={'1':''}
              >
              >everything works fine.
              >
              >Best regards,
              >Marcin
              >
              >
              >
              >
              >
              >
              >--
              >You received this message from the "vim_dev" maillist.
              >Do not top-post! Type your reply below the text you are replying to.
              >For more information, visit http://www.vim.org/maillist.php

              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php
            • ZyX
              ... Agreed here. # HG changeset patch # User ZyX # Date 1346519818 -14400 # Node ID fac3b8f96d30583e39f6f46ac0d63269a859a164 # Parent
              Message 6 of 11 , Sep 1, 2012
              View Source
              • 0 Attachment
                суббота, 1 сентября 2012 г., 20:32:39 UTC+4 пользователь Marcin Szamotulski написал:
                > I looked into the documentation in the mercurial (I should check it
                > before but my vim has bit old runtime tree) and it suggests that the
                > keys are converted to strings:
                >
                > Dictionaries are represented as Vim |Dictionary| type with
                > keys converted to strings.
                >
                > So or E859 is a bug or the documentation should be changed.

                Agreed here.

                # HG changeset patch
                # User ZyX <kp-pav@...>
                # Date 1346519818 -14400
                # Node ID fac3b8f96d30583e39f6f46ac0d63269a859a164
                # Parent 0fa6134265ec58e8dba97cdb8e2a6730e05361df
                Fixed documentation for pyeval()

                diff -r 0fa6134265ec -r fac3b8f96d30 runtime/doc/eval.txt
                --- a/runtime/doc/eval.txt Sat Sep 01 18:08:04 2012 +0400
                +++ b/runtime/doc/eval.txt Sat Sep 01 21:16:33 2012 +0400
                @@ -4635,8 +4635,8 @@
                Numbers and strings are returned as they are (strings are
                copied though).
                Lists are represented as Vim |List| type.
                - Dictionaries are represented as Vim |Dictionary| type with
                - keys converted to strings.
                + Dictionaries are represented as Vim |Dictionary| type,
                + non-string keys result in error.
                {only available when compiled with the |+python| feature}

                *E726* *E727*

                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php
              • ZyX
                ... I used json.dumps and a self-written dumper for this job. The above code is one of the worst ideas that could go into your head: 1. (string %
                Message 7 of 11 , Sep 1, 2012
                View Source
                • 0 Attachment
                  > Prior to pyeval() function I was using
                  >
                  > vim.command("let dict=%s" % python_dict)

                  I used json.dumps and a self-written dumper for this job. The above code is one of the worst ideas that could go into your head:

                  1. (string % tuple_or_value) notation is deprecated.
                  2. “"%s" % {u'«»': 1}” produces “{u'\xab\xbb': 1}” which can’t be handled in vim.
                  3. “"%s" % {'«»': 1}” produces “{'\xc2\xab\xc2\xbb': 1}” which is not handled by vim in a proper way.
                  4. There is a bunch of other things calling __repl__ on which will result in something vim won’t understand, but in this case vim won’t even understand simple unicode or utf-8 encoded strings as shown above.

                  json.dumps minimizes the amount of problematic things (including converting keys to strings) and throws an exception if it can’t handle something, my interface does this as well. I do not do some type conversions that json.dumps does, json.dumps fails to handle some things which I handle (“pyeval('vim.current.buffer')” produces same output as “getline(1, '$')”, but “json.dumps(vim.current.buffer)” fails).

                  Self-written dumper was used to get binary data: “json.dumps('\x80')” throws an exception while vim is fine with just having it embedded inside a string.

                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php
                • Sean Estabrooks
                  ... This seems a little harsh. The above code is convenient and rather straightforward. Shouldn t we find a way to make it work? ... Obviously it would be much
                  Message 8 of 11 , Sep 1, 2012
                  View Source
                  • 0 Attachment
                    > > Prior to pyeval() function I was using
                    > >
                    > > vim.command("let dict=%s" % python_dict)
                    >
                    > I used json.dumps and a self-written dumper for this job. The above code is one of the worst ideas that could go into your head:

                    This seems a little harsh. The above code is convenient and rather straightforward. Shouldn't we find a way to make it work?

                    >
                    > 1. (string % tuple_or_value) notation is deprecated.
                    > 2. “"%s" % {u'«»': 1}” produces “{u'\xab\xbb': 1}” which can’t be handled in vim.
                    > 3. “"%s" % {'«»': 1}” produces “{'\xc2\xab\xc2\xbb': 1}” which is not handled by vim in a proper way.
                    > 4. There is a bunch of other things calling __repl__ on which will result in something vim won’t understand, but in this case vim won’t even understand simple unicode or utf-8 encoded strings as shown above.
                    >
                    > json.dumps minimizes the amount of problematic things (including converting keys to strings) and throws an exception if it can’t handle something, my interface does this as well. I do not do some type conversions that json.dumps does, json.dumps fails to handle some things which I handle (“pyeval('vim.current.buffer')” produces same output as “getline(1, '$')”, but “json.dumps(vim.current.buffer)” fails).
                    >
                    > Self-written dumper was used to get binary data: “json.dumps('\x80')” throws an exception while vim is fine with just having it embedded inside a string.
                    >

                    Obviously it would be much better if everyone who is using python inside of Vim doesn't have to write their own custom code to transfer these values.  Do you have a plan in this regard or is this an open problem?

                    Thanks,
                    Sean



                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php
                  • Marcin Szamotulski
                    ... This the purpose of pyeval() and py3eval() functions which came with a recent patch (by Zyx). Thank you Zyx for showing me the weak points. I only used
                    Message 9 of 11 , Sep 1, 2012
                    View Source
                    • 0 Attachment
                      On 16:06 Sat 01 Sep , Sean Estabrooks wrote:
                      >
                      >
                      >
                      > > > Prior to pyeval() function I was using
                      > > >
                      > > > vim.command("let dict=%s" % python_dict)
                      > >
                      > > I used json.dumps and a self-written dumper for this job. The above code is one of the worst ideas that could go into your head:
                      >
                      > This seems a little harsh. The above code is convenient and rather straightforward. Shouldn't we find a way to make it work?
                      >
                      > >
                      > > 1. (string % tuple_or_value) notation is deprecated.
                      > > 2. “"%s" % {u'«»': 1}” produces “{u'\xab\xbb': 1}” which can’t be handled in vim.
                      > > 3. “"%s" % {'«»': 1}” produces “{'\xc2\xab\xc2\xbb': 1}” which is not handled by vim in a proper way.
                      > > 4. There is a bunch of other things calling __repl__ on which will result in something vim won’t understand, but in this case vim won’t even understand simple unicode or utf-8 encoded strings as shown above.
                      > >
                      > > json.dumps minimizes the amount of problematic things (including converting keys to strings) and throws an exception if it can’t handle something, my interface does this as well. I do not do some type conversions that json.dumps does, json.dumps fails to handle some things which I handle (“pyeval('vim.current.buffer')” produces same output as “getline(1, '$')”, but “json.dumps(vim.current.buffer)” fails).
                      > >
                      > > Self-written dumper was used to get binary data: “json.dumps('\x80')” throws an exception while vim is fine with just having it embedded inside a string.
                      > >
                      >
                      > Obviously it would be much better if everyone who is using python inside of Vim doesn't have to write their own custom code to transfer these values.  Do you have a plan in this regard or is this an open problem?
                      >
                      > Thanks,
                      > Sean

                      This the purpose of pyeval() and py3eval() functions which came with
                      a recent patch (by Zyx).

                      Thank you Zyx for showing me the weak points. I only used this for ASCII
                      and integers, though at some point this would lead to a problem.

                      Best,
                      Marcin

                      --
                      You received this message from the "vim_dev" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php
                    • Sean Estabrooks
                      ... Ah, looks good.  Thanks for setting me straight. Sean -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the
                      Message 10 of 11 , Sep 1, 2012
                      View Source
                      • 0 Attachment
                        ----------------------------------------
                        > Date: Sun, 2 Sep 2012 00:54:42 +0100
                        > From: mszamot@...
                        > To: vim_dev@...
                        > Subject: Re: pyeval() error
                        >
                        > On 16:06 Sat 01 Sep , Sean Estabrooks wrote:
                        ...
                        >> Obviously it would be much better if everyone who is using python inside of Vim doesn't have to write their own custom code to transfer these values. Do you have a plan in this regard or is this an open problem?
                        >>
                        >
                        > This the purpose of pyeval() and py3eval() functions which came with
                        > a recent patch (by Zyx).
                        >
                        > Thank you Zyx for showing me the weak points. I only used this for ASCII
                        > and integers, though at some point this would lead to a problem.

                        Ah, looks good.  Thanks for setting me straight.

                        Sean




                        --
                        You received this message from the "vim_dev" maillist.
                        Do not top-post! Type your reply below the text you are replying to.
                        For more information, visit http://www.vim.org/maillist.php
                      • ZyX
                        ... Not exactly. Try to transfer a few hundred KiBs with json.dumps and with my interface and you will see the difference. Self-written dumper is much, much
                        Message 11 of 11 , Sep 2, 2012
                        View Source
                        • 0 Attachment
                          > This the purpose of pyeval() and py3eval() functions which came with
                          > a recent patch (by Zyx).

                          Not exactly. Try to transfer a few hundred KiBs with json.dumps and with my interface and you will see the difference. Self-written dumper is much, much slower as it is not written in C and it was necessary to make it able to get even more information from time to time. That was the real purpose as dumper is trivial and easy to write (though vim.bindeval interface solves custom dumpers problem as well).

                          By the way, why almost everybody says that it were pyeval functions that were added in first place? Main change is vim.bindeval as you can write code like

                          def return_to_vim(value):
                          if hasattr(vim, 'bindeval'):
                          vim.bindeval('d')['result']=value
                          else:
                          vim.eval('extend(d, {"result": '+dumps(value)+'})')
                          and be compatible with old versions of vim. pyeval() does not allow you to do it so easily (though I can imagine Pyeval function that works in old vims as well, it is more to type). pyeval() is slightly faster, but it is negligible and constant difference.

                          --
                          You received this message from the "vim_dev" maillist.
                          Do not top-post! Type your reply below the text you are replying to.
                          For more information, visit http://www.vim.org/maillist.php
                        Your message has been successfully submitted and would be delivered to recipients shortly.