Loading ...
Sorry, an error occurred while loading the content.
 

Re: Python items in the todo list

Expand Messages
  • Bram Moolenaar
    ... Interesting. So we would have a $VIMRUNTIME/python directory and we can put Python scripts there that we include with the distribution. I m sure it is
    Message 1 of 26 , May 17, 2013
      Xavier de Gaye wrote:

      > >> On Wed, May 15, 2013 at 8:51 PM, Bram Moolenaar wrote:
      > >> >> >":python os.chdir('/tmp')" makes short buffer names invalid. (Xavier de Gaye)
      > >> >> >Check directory and call shorten_fnames()?
      > >> >>
      > >> >> Probably not only the python problem.
      > >> >
      > >> > I wonder if there is a hook inside Python, so that we can do the
      > >> > equivalent of ":cd" in Vim, handling the side effects.
      > >>
      > >>
      > >> Indeed, a hook to chdir can be made with the following code in
      > >> a module named vim_hook:
      > >>
      > >> ########### vim_hook.py ###########
      > >> import vim
      > >> import os
      > >>
      > >> _chdir = os.chdir
      > >>
      > >> def chdir(path):
      > >> _chdir(path)
      > >> vim.command("cd " + path)
      > >>
      > >> os.chdir = chdir
      > >> ###########
      > >>
      > >> This module is imported on vim startup with:
      > >>
      > >> PyRun_SimpleString("import vim_hook");
      > >>
      > >>
      > >> To test that this fixes os.chdir, run the following commands after
      > >> having copied the above vim_hook.py file in the current directory:
      > >>
      > >> :py3 import sys; sys.path[:1] = ['.']; import vim_hook
      > >> :py3 import os; os.chdir("/tmp")
      > >
      > > OK, but we want this to work without any commands being used in a
      > > script. It should be done when initalizing Python.
      >
      > This is what I meant.
      >
      > The attached patch does this.
      >
      > A new directory named 'python' is added to the vim runtime files.
      > This directory contains at the moment a _vim_python.py file that is
      > imported when initalizing Python after the python directory pathname
      > has been added to sys.path.
      >
      > _vim_python.py is meant to contain (or to import the other components
      > of) the python part of the vim-python interface.

      Interesting. So we would have a $VIMRUNTIME/python directory and we can
      put Python scripts there that we include with the distribution.

      I'm sure it is only a small step that users will ask to have a python
      directory in ~/.vim/python. Or more general, using 'runtimepath'.

      Then a plugin does not require to have its Python code in between a
      :python << EOF and EOF. That would be a lot nicer, right?

      What do others think?

      --
      BROTHER MAYNARD: Armaments Chapter Two Verses Nine to Twenty One.
      ANOTHER MONK: And St. Attila raised his hand grenade up on high saying "O
      Lord bless this thy hand grenade that with it thou mayest
      blow thine enemies to tiny bits, in thy mercy. "and the Lord
      did grin and people did feast upon the lambs and sloths and
      carp and anchovies and orang-utans and breakfast cereals and
      fruit bats and...
      BROTHER MAYNARD: Skip a bit brother ...
      "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

      /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
      /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
      \\\ an exciting new programming language -- http://www.Zimbu.org ///
      \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php

      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
    • ZyX
      ... Automatically adding all items from `&rtp` to sys.path would be good. It (and python3/) is already a standard directory in frawor for python files. But I
      Message 2 of 26 , May 17, 2013
        > Interesting. So we would have a $VIMRUNTIME/python directory and we can
        > put Python scripts there that we include with the distribution.
        >
        > I'm sure it is only a small step that users will ask to have a python
        > directory in ~/.vim/python. Or more general, using 'runtimepath'.
        >
        > Then a plugin does not require to have its Python code in between a
        > :python << EOF and EOF. That would be a lot nicer, right?
        >
        > What do others think?

        Automatically adding all items from `&rtp` to sys.path would be good. It (and python3/) is already a standard directory in frawor for python files.

        But I would vote against the patch to os.chdir implemented in python and (as there is no better variant) for the solution based on comparing current directories before and after `os.chdir`:

        1. Implementation based on comparing current directories can be written once and easily applied to all other interfaces.
        2. `os.chdir` is most common, but not the only way to change directories from python: there are also at least `posix.chdir` and calls to libc (e.g. indirectly from some bindings or directly using ctypes, though I can’t imagine why the latter may be required).
        3. Proposed implementation will break once somebody deletes `os` from sys.modules for some reason (I used to use this variant to reset `os` module after mocking its methods for testing purposes; though as for all non-builtin modules touching `sys.modules`).

        About the implementation of `&rtp` handling: you can add there to `sys.path` a special path like `'_vim_runtimepaths_'` and add hook to `sys.path_hooks` that can handle it. Requires dropping python 2.2 support (`path_hooks` were added in python 2.3), but you won’t then need to hack `options.c` to add or remove appropriate paths from `sys.path` when changing `&rtp`.

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php

        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
      • Bram Moolenaar
        ... I think this implies we need to get and save the current directory for every Python command that is executed. Doesn t this add quite a bit of overhead? I
        Message 3 of 26 , May 17, 2013
          ZyX wrote:

          > > Interesting. So we would have a $VIMRUNTIME/python directory and we can
          > > put Python scripts there that we include with the distribution.
          > >
          > > I'm sure it is only a small step that users will ask to have a python
          > > directory in ~/.vim/python. Or more general, using 'runtimepath'.
          > >
          > > Then a plugin does not require to have its Python code in between a
          > > :python << EOF and EOF. That would be a lot nicer, right?
          > >
          > > What do others think?
          >
          > Automatically adding all items from `&rtp` to sys.path would be good.
          > It (and python3/) is already a standard directory in frawor for python
          > files.
          >
          > But I would vote against the patch to os.chdir implemented in python
          > and (as there is no better variant) for the solution based on
          > comparing current directories before and after `os.chdir`:
          >
          > 1. Implementation based on comparing current directories can be
          > written once and easily applied to all other interfaces.
          > 2. `os.chdir` is most common, but not the only way to change
          > directories from python: there are also at least `posix.chdir` and
          > calls to libc (e.g. indirectly from some bindings or directly using
          > ctypes, though I can’t imagine why the latter may be required).
          > 3. Proposed implementation will break once somebody deletes `os` from
          > sys.modules for some reason (I used to use this variant to reset `os`
          > module after mocking its methods for testing purposes; though as for
          > all non-builtin modules touching `sys.modules`).

          I think this implies we need to get and save the current directory for
          every Python command that is executed. Doesn't this add quite a bit of
          overhead? I originally asked for some kind of hook or callback, so that
          we can do something when Python changes directory at a lower level.

          I suppose we do need to wait with the Vim implementation of :cd until
          the Python command finishes. Thus os.chdir() would set a flag. Then
          when the Python command restores the directory we don't need to do
          anything. I would expect this to be quite common:
          saved_curdir = os.cwd()
          os.chdir(somewhere)
          ... do something with curdir ..
          os.chdir(saved_curdir)

          > About the implementation of `&rtp` handling: you can add there to
          > `sys.path` a special path like `'_vim_runtimepaths_'` and add hook to
          > `sys.path_hooks` that can handle it. Requires dropping python 2.2
          > support (`path_hooks` were added in python 2.3), but you won’t then
          > need to hack `options.c` to add or remove appropriate paths from
          > `sys.path` when changing `&rtp`.

          Do we need this kind of magic? Updating sys.path is much simpler and
          also does not hide what we are doing (in case someone is debugging
          what's going on). The only trick we need is when the user changes
          'runtimepath'. But that would happen very infrequently.

          --
          Arthur pulls Pin out. The MONK blesses the grenade as ...
          ARTHUR: (quietly) One, two, five ...
          GALAHAD: Three, sir!
          ARTHUR: Three.
          "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

          /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
          /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
          \\\ an exciting new programming language -- http://www.Zimbu.org ///
          \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

          --
          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_dev" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • Xavier de Gaye
          ... `os.chdir` and `posix.chdir` are two names that are bound to the same ... True `posix.chdir` is low level and is never used. The python documentation on
          Message 4 of 26 , May 18, 2013
            On Fri, May 17, 2013 at 9:07 PM, ZyX wrote:
            >> Interesting. So we would have a $VIMRUNTIME/python directory and we can
            >> put Python scripts there that we include with the distribution.
            >>
            >> I'm sure it is only a small step that users will ask to have a python
            >> directory in ~/.vim/python. Or more general, using 'runtimepath'.
            >>
            >> Then a plugin does not require to have its Python code in between a
            >> :python << EOF and EOF. That would be a lot nicer, right?
            >>
            >> What do others think?
            >
            > Automatically adding all items from `&rtp` to sys.path would be
            > good. It (and python3/) is already a standard directory in frawor
            > for python files.
            >
            > But I would vote against the patch to os.chdir implemented in python
            > and (as there is no better variant) for the solution based on
            > comparing current directories before and after `os.chdir`:
            >
            > 1. Implementation based on comparing current directories can be
            > written once and easily applied to all other interfaces.
            > 2. `os.chdir` is most common, but not the only way to change
            > directories from python: there are also at least `posix.chdir` and


            `os.chdir` and `posix.chdir` are two names that are bound to the same
            built-in function, as you can check with:

            >>> import os, posix
            >>> os.chdir is posix.chdir
            True

            `posix.chdir` is low level and is never used. The python documentation
            on 'posix' states:

            "Do not import this module directly. Instead, import the module os".

            `posix.chdir` is not even listed in the python documentation.


            > calls to libc (e.g. indirectly from some bindings or directly using
            > ctypes, though I can’t imagine why the latter may be required).


            This is kind of a red herring as the same issue already exists with
            the vimL libcall() function.


            > 3. Proposed implementation will break once somebody deletes `os`
            > from sys.modules for some reason (I used to use this variant to
            > reset `os` module after mocking its methods for testing purposes;
            > though as for all non-builtin modules touching `sys.modules`).


            It is not good practice to import a module twice (unless you own it
            and thus, know what you are doing) because importing a module may have
            side effects: when you import a module, you execute all the statements
            within the module, and executing them twice might not have been
            expected by the author of the module.

            If you mock some 'os' methods for testing purposes, then, when you are
            done with it, you must restore those methods instead of deleting the
            module from sys.modules and re-importing it a second time which is not
            safe.


            > About the implementation of `&rtp` handling: you can add there to
            > `sys.path` a special path like `'_vim_runtimepaths_'` and add hook
            > to `sys.path_hooks` that can handle it. Requires dropping python 2.2
            > support (`path_hooks` were added in python 2.3), but you won’t then
            > need to hack `options.c` to add or remove appropriate paths from
            > `sys.path` when changing `&rtp`.

            --
            Xavier

            Les Chemins de Lokoti: http://lokoti.alwaysdata.net

            --
            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php

            ---
            You received this message because you are subscribed to the Google Groups "vim_dev" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
          • ZyX
            ... This does not matter. When you assign to `os.chdir` they are no longer identical. ... libcall() can use the same function for checking for os.chdir. ...
            Message 5 of 26 , May 18, 2013
              >>> Interesting. So we would have a $VIMRUNTIME/python directory and we can
              >>> put Python scripts there that we include with the distribution.
              >>>
              >>> I'm sure it is only a small step that users will ask to have a python
              >>> directory in ~/.vim/python. Or more general, using 'runtimepath'.
              >>>
              >>> Then a plugin does not require to have its Python code in between a
              >>> :python << EOF and EOF. That would be a lot nicer, right?
              >>>
              >>> What do others think?
              >>
              >> Automatically adding all items from `&rtp` to sys.path would be
              >> good. It (and python3/) is already a standard directory in frawor
              >> for python files.
              >>
              >> But I would vote against the patch to os.chdir implemented in python
              >> and (as there is no better variant) for the solution based on
              >> comparing current directories before and after `os.chdir`:
              >>
              >> 1. Implementation based on comparing current directories can be
              >> written once and easily applied to all other interfaces.
              >> 2. `os.chdir` is most common, but not the only way to change
              >> directories from python: there are also at least `posix.chdir` and
              >
              >
              >`os.chdir` and `posix.chdir` are two names that are bound to the same
              >built-in function, as you can check with:

              This does not matter. When you assign to `os.chdir` they are no longer
              identical.

              > >>> import os, posix
              > >>> os.chdir is posix.chdir
              > True
              >
              >`posix.chdir` is low level and is never used. The python documentation
              >on 'posix' states:
              >
              > "Do not import this module directly. Instead, import the module os".
              >
              >`posix.chdir` is not even listed in the python documentation.
              >
              >
              >> calls to libc (e.g. indirectly from some bindings or directly using
              >> ctypes, though I can’t imagine why the latter may be required).
              >
              >
              >This is kind of a red herring as the same issue already exists with
              >the vimL libcall() function.

              libcall() can use the same function for checking for os.chdir.

              >> 3. Proposed implementation will break once somebody deletes `os`
              >> from sys.modules for some reason (I used to use this variant to
              >> reset `os` module after mocking its methods for testing purposes;
              >> though as for all non-builtin modules touching `sys.modules`).
              >
              >
              >It is not good practice to import a module twice (unless you own it
              >and thus, know what you are doing) because importing a module may have
              >side effects: when you import a module, you execute all the statements
              >within the module, and executing them twice might not have been
              >expected by the author of the module.

              `os` seems to be safe regarding the issue.

              >If you mock some 'os' methods for testing purposes, then, when you are
              >done with it, you must restore those methods instead of deleting the
              >module from sys.modules and re-importing it a second time which is not
              >safe.

              Yes, I changed the implementation immediately after I found that removing from
              `sys.modules` may introduce strange TypeErrors (objects in module got assigned
              None). I have never seen such behavior for C modules though, only for python
              ones.

              >> About the implementation of `&rtp` handling: you can add there to
              >> `sys.path` a special path like `'_vim_runtimepaths_'` and add hook
              >> to `sys.path_hooks` that can handle it. Requires dropping python 2.2
              >> support (`path_hooks` were added in python 2.3), but you won’t then
              >> need to hack `options.c` to add or remove appropriate paths from
              >> `sys.path` when changing `&rtp`.

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
            • ZyX
              ... There are two other big advantages of this “magic”: user did not loose control over `sys.path` and it can be written in pure python. All advantages in
              Message 6 of 26 , May 18, 2013
                >Do we need this kind of magic? Updating sys.path is much simpler and
                >also does not hide what we are doing (in case someone is debugging
                >what's going on). The only trick we need is when the user changes
                >'runtimepath'. But that would happen very infrequently.

                There are two other big advantages of this “magic”: user did not loose control
                over `sys.path` and it can be written in pure python. All advantages in a list:
                it …

                1. makes it possible to specify which directories should be searched before
                plugin ones and which after;
                2. makes it possible to remove vim paths from sys.path completely and not ever
                have problems with vim adding them again;
                3. can be written in pure python and fallback implementation for old vims (the
                ones even without vim.eval) can be created;
                4. makes hacking non-if_py* C files not needed.

                There is a reason for coding it in C though (and porting @Xavier de Gaye file to C as well): this makes vim core functionality independent of presence of runtime files. There is a reason for not coding it in C as well: it is faster to write in python and even being written in python this will not have problems with performance (it’ll in any case waste much time on IO).

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php

                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
              • Xavier de Gaye
                ... I think that it is reasonable to answer to anyone that complains something is broken after he deletes a module from sys.modules and re-import it again:
                Message 7 of 26 , May 19, 2013
                  On Sat, May 18, 2013 at 4:12 PM, ZyX wrote:
                  >>>> Interesting. So we would have a $VIMRUNTIME/python directory and we can
                  >>>> put Python scripts there that we include with the distribution.
                  >>>>
                  >>>> I'm sure it is only a small step that users will ask to have a python
                  >>>> directory in ~/.vim/python. Or more general, using 'runtimepath'.
                  >>>>
                  >>>> Then a plugin does not require to have its Python code in between a
                  >>>> :python << EOF and EOF. That would be a lot nicer, right?
                  >>>>
                  >>>> What do others think?
                  >>>
                  >>> Automatically adding all items from `&rtp` to sys.path would be
                  >>> good. It (and python3/) is already a standard directory in frawor
                  >>> for python files.
                  >>>
                  >>> But I would vote against the patch to os.chdir implemented in python
                  >>> and (as there is no better variant) for the solution based on
                  >>> comparing current directories before and after `os.chdir`:
                  >>>
                  >>> 1. Implementation based on comparing current directories can be
                  >>> written once and easily applied to all other interfaces.
                  >>> 2. `os.chdir` is most common, but not the only way to change
                  >>> directories from python: there are also at least `posix.chdir` and
                  >>
                  >>
                  >>`os.chdir` and `posix.chdir` are two names that are bound to the same
                  >>built-in function, as you can check with:
                  >
                  > This does not matter. When you assign to `os.chdir` they are no longer
                  > identical.
                  >
                  >> >>> import os, posix
                  >> >>> os.chdir is posix.chdir
                  >> True
                  >>
                  >>`posix.chdir` is low level and is never used. The python documentation
                  >>on 'posix' states:
                  >>
                  >> "Do not import this module directly. Instead, import the module os".
                  >>
                  >>`posix.chdir` is not even listed in the python documentation.
                  >>
                  >>
                  >>> calls to libc (e.g. indirectly from some bindings or directly using
                  >>> ctypes, though I can’t imagine why the latter may be required).
                  >>
                  >>
                  >>This is kind of a red herring as the same issue already exists with
                  >>the vimL libcall() function.
                  >
                  > libcall() can use the same function for checking for os.chdir.
                  >
                  >>> 3. Proposed implementation will break once somebody deletes `os`
                  >>> from sys.modules for some reason (I used to use this variant to
                  >>> reset `os` module after mocking its methods for testing purposes;
                  >>> though as for all non-builtin modules touching `sys.modules`).
                  >>
                  >>
                  >>It is not good practice to import a module twice (unless you own it
                  >>and thus, know what you are doing) because importing a module may have
                  >>side effects: when you import a module, you execute all the statements
                  >>within the module, and executing them twice might not have been
                  >>expected by the author of the module.
                  >
                  > `os` seems to be safe regarding the issue.
                  >
                  >>If you mock some 'os' methods for testing purposes, then, when you are
                  >>done with it, you must restore those methods instead of deleting the
                  >>module from sys.modules and re-importing it a second time which is not
                  >>safe.
                  >
                  > Yes, I changed the implementation immediately after I found that removing from
                  > `sys.modules` may introduce strange TypeErrors (objects in module got assigned
                  > None). I have never seen such behavior for C modules though, only for python
                  > ones.


                  I think that it is reasonable to answer to anyone that complains
                  something is broken after he deletes a module from sys.modules and
                  re-import it again:

                  "Please don't do that, if you really need to do it, then you are
                  on your own".

                  For example with the 'sys' module in the vim-python interface ('sys' is
                  a C extension module by the way):

                  :py3 import sys; del sys.modules['sys']
                  :py3 import sys
                  :py3 print(sys.stdout)
                  Traceback (most recent call last):
                  File "<string>", line 1, in <module>
                  AttributeError: 'module' object has no attribute 'stdout'
                  Press ENTER or type command to continue

                  Re-binding a name such as done in the patch with os.chdir is actually
                  quite common. It is even already done in the current vim-python
                  interface with sys.stdout and sys.stderr (see PythonIO_Init_io()). A
                  vim user writing a python program with the vim-python interface may
                  also re-bind sys.stdout and sys.stderr to implement i/o redirection,
                  but he is expected to restore the original bindings after he is done
                  and certainly must not delete the 'sys' module and re-import it again.

                  --
                  Xavier

                  Les Chemins de Lokoti: http://lokoti.alwaysdata.net

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php

                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                Your message has been successfully submitted and would be delivered to recipients shortly.