Loading ...
Sorry, an error occurred while loading the content.
 

Re: Dynamically loading Python library

Expand Messages
  • Jason Foreman
    Hi Björn, ... I believe you must have installed the version of Python that your Vim distribution was built with. For standard builds I think this is 2.3.
    Message 1 of 15 , Nov 1, 2008
      Hi Björn,

      On Nov 1, 2008, at 11:57 AM, björn wrote:

      > How is this solved on Windows?

      I believe you must have installed the version of Python that your Vim
      distribution was built with. For standard builds I think this is
      2.3. Cream builds use 2.5 or maybe 2.4. I can verify this on Monday
      at work.

      > Please, even if nobody knows the answer to this question I urge you to

      > look into it and help me out, else I will be forced to continue
      > distributing MacVim with Python 2.3.


      I don't have an answer, but I'm willing to help if I can. I'm not
      100% sure there will be a solution short of having separate Leopard
      and Tiger builds, but it's worth trying.

      One thing that might work is to use "Py_InitModule3" instead of
      "Py_InitModule4" in if_python.c:2361. The former does not take an API
      version parameter, so that check won't be performed. Even the online
      Python docs [1] say that use of "Py_InitModule3" is preferred in most
      cases. I haven't got Tiger setup at the moment to try this out. I'll
      work on getting Tiger going this weekend so I can try your patch.

      Another thing to try would be downloading the Python 2.3 sources and
      using the headers from there when building, instead of the Python 2.5
      headers included with Leopard. This would ensure the
      PYTHON_API_VERSION matches. But I'm not sure if this would fix the
      problem or just invert it (no warn on Tiger, warn on Leopard).

      Jason

      [1] http://www.python.org/doc/2.5.2/api/allocating-objects.html
    • Ben Schmidt
      ... Hehe. I could have told you that and saved you the trouble. It s come up on one of the Vim mailing lists before. It s not solved on Windows, but IIRC,
      Message 2 of 15 , Nov 2, 2008
        > Now, I have managed to get Python to load dynamically and have tested
        > on Leopard/Intel and Tiger/PPC and on the former it loads 2.5, on the
        > latter 2.3. Yay! Or not. It turns out that there was an API change
        > between 2.3 and 2.5 and this is checked via a define
        > PYTHON_API_VERSION which is defined inside a Python header
        > (modsupport.h).

        Hehe. I could have told you that and saved you the trouble. It's come up
        on one of the Vim mailing lists before. It's not solved on Windows, but
        IIRC, gives even more drastic problems than it seems to on the Mac. :-\

        Unless you know exactly what the API changes are, you're pretty much
        stuffed. Any change in a structure Vim uses or the signature or return
        type of a function Vim calls, or a constant/macro definition Vim uses,
        and things will go wrong if the wrong version is dynamically loaded. In
        a sense, the warning is the least of your worries.

        You could ask the Python guys to try to find out exactly what the API
        changes are and whether they apply to Vim or not, but this could be
        difficult to ascertain--these things tend to be a bit of a haystack. On
        the other hand, it could be the change is very simple and irrelevant,
        and you can just somehow disable the warning--if this is a likely
        scenario, the Python guys almost certainly will have provided a way to
        disable it.

        If it can't be ascertained whether change is irrelevant, or it can be
        ascertained that it *is* relevant, it would be technically possible to
        make a Vim that worked with both API versions, but you'd have to
        basically duplicate if_python.c, change all the function names so there
        were two versions of each, compile one against the 2.3 API and another
        against the 2.5 API and then have a runtime check to decide which
        version of the functions to call. It would be quite a refactoring job,
        and I would say not worth the effort.

        One definite solution, though not ideal, but which is easy to set up for
        building, as well as easy for users to understand: have separate MacVim
        builds for Tiger and Leopard. Stacks of apps do it, and Mac users are
        fairly used to downloading the version appropriate to their OS version.

        Ben.




        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_mac" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • Ben Schmidt
        ... I m not so sure about that. I do use :python, but only indirectly, by using plugins that use it. As such, if it s good enough for the plugin, it s good
        Message 3 of 15 , Nov 2, 2008
          > If Python is dynamically loaded, then people running Python 2.3 on
          > Tiger don't pay any cost until they use :python. I'm inclined to think
          > that most people who want to use :python have Python 2.5 or 2.6
          > installed.

          I'm not so sure about that. I do use :python, but only indirectly, by
          using plugins that use it. As such, if it's good enough for the plugin,
          it's good enough for me, and I won't bother installing a later version
          than the one the OS provides unless something really requires it. I'm
          certainly not going to install one just for the love of Python or of
          being on the cutting edge. And to be honest, I wouldn't be sure that if
          I did install a later one, that would be the one MacVim would find first
          in the path. I suspect there are other users like me. In fact, I suspect
          more people use :python via plugins than use it directly in their own
          scripts or from the commandline.

          None of this actually affects me, as I compile MacVim from source
          myself, linking statically against whatever Python it finds. But I'm
          just pointing out that I expect it would affect others.

          Ben.




          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_mac" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • George V. Reilly
          2008/11/2 Ben Schmidt ... When I ported the Vim Python interface code to Win64, I looked around for Vim plugins to exercise it,
          Message 4 of 15 , Nov 2, 2008
            2008/11/2 Ben Schmidt <mail_ben_schmidt@...>
            > If Python is dynamically loaded, then people running Python 2.3 on
            > Tiger don't pay any cost until they use :python. I'm inclined to think
            > that most people who want to use :python have Python 2.5 or 2.6
            > installed.

            I'm not so sure about that. I do use :python, but only indirectly, by
            using plugins that use it.

            When I ported the Vim Python interface code to Win64, I looked around for Vim plugins to exercise it, and I had a hard time finding any. That was about a year ago. Maybe people have started using it more.

            Anyway, separate builds for Tiger and Leopard is probably the best way to go.
            --
            /George V. Reilly  george@...
            http://www.georgevreilly.com/blog  http://blogs.cozi.com/tech

            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_mac" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---

          • Ben Schmidt
            ... I expect there are more plugins that use it on non-Windows systems. I think I have a couple installed that use it, but certainly the main one is
            Message 5 of 15 , Nov 2, 2008
              George V. Reilly wrote:
              > 2008/11/2 Ben Schmidt <mail_ben_schmidt@...
              > <mailto:mail_ben_schmidt@...>>
              >
              > > If Python is dynamically loaded, then people running Python 2.3 on
              > > Tiger don't pay any cost until they use :python. I'm inclined to
              > think
              > > that most people who want to use :python have Python 2.5 or 2.6
              > > installed.
              >
              > I'm not so sure about that. I do use :python, but only indirectly, by
              > using plugins that use it.
              >
              > When I ported the Vim Python interface code to Win64, I looked around
              > for Vim plugins to exercise it, and I had a hard time finding any. That
              > was about a year ago. Maybe people have started using it more.

              I expect there are more plugins that use it on non-Windows systems.

              I think I have a couple installed that use it, but certainly the main
              one is

              http://www.vim.org/scripts/script.php?script_id=1929

              which I use from time to time to debug PHP. I first started using it
              about a year ago too. It's the sort of thing that can make a big
              difference to your coding experience, requires Python to be working, but
              doesn't need the latest Python version. And certainly the user doesn't
              want to be fiddling with any of the Python code or other Python-related
              problems (like me, may not even know any Python). They already have
              problems in their own code...

              Grins,

              Ben.




              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_mac" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            • björn
              ... I don t think it is a feasible solution...e.g. what happens the next time the API changes? No, I think loading Python dynamically is out of the question.
              Message 6 of 15 , Nov 3, 2008
                2008/11/3 Ben Schmidt <mail_ben_schmidt@...>:
                >
                > You could ask the Python guys to try to find out exactly what the API
                > changes are and whether they apply to Vim or not, but this could be
                > difficult to ascertain--these things tend to be a bit of a haystack. On
                > the other hand, it could be the change is very simple and irrelevant,
                > and you can just somehow disable the warning--if this is a likely
                > scenario, the Python guys almost certainly will have provided a way to
                > disable it.

                I don't think it is a feasible solution...e.g. what happens the next
                time the API changes? No, I think loading Python dynamically is out
                of the question. :(

                > If it can't be ascertained whether change is irrelevant, or it can be
                > ascertained that it *is* relevant, it would be technically possible to
                > make a Vim that worked with both API versions, but you'd have to
                > basically duplicate if_python.c, change all the function names so there
                > were two versions of each, compile one against the 2.3 API and another
                > against the 2.5 API and then have a runtime check to decide which
                > version of the functions to call. It would be quite a refactoring job,
                > and I would say not worth the effort.

                Ugh, yeah, that's no good either.

                > One definite solution, though not ideal, but which is easy to set up for
                > building, as well as easy for users to understand: have separate MacVim
                > builds for Tiger and Leopard. Stacks of apps do it, and Mac users are
                > fairly used to downloading the version appropriate to their OS version.

                It seems this is the only way to go then. But...I wonder if it really
                is worth my time making two distributions just so a few users get to
                use Python 2.5 instead of 2.3. I think I need a better reason than
                that for something as drastic as distributing two different versions
                of the binary. Maybe I underestimate the number of users who would
                benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                Thanks to everybody for the feedback,
                Björn

                --~--~---------~--~----~------------~-------~--~----~
                You received this message from the "vim_mac" maillist.
                For more information, visit http://www.vim.org/maillist.php
                -~----------~----~----~----~------~----~------~--~---
              • Jason Foreman
                On Mon, Nov 3, 2008 at 11:55 AM, björn wrote: 2008/11/3 Ben Schmidt : have separate MacVim
                Message 7 of 15 , Nov 3, 2008
                  On Mon, Nov 3, 2008 at 11:55 AM, björn <bjorn.winckler@...> wrote:
                  > 2008/11/3 Ben Schmidt <mail_ben_schmidt@...>:
                  >> have separate MacVim builds for Tiger and Leopard.
                  >
                  > It seems this is the only way to go then. But...I wonder if it really
                  > is worth my time making two distributions just so a few users get to
                  > use Python 2.5 instead of 2.3. I think I need a better reason than
                  > that for something as drastic as distributing two different versions
                  > of the binary. Maybe I underestimate the number of users who would
                  > benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                  FWIW, I've been toying with the idea of making my own Leopard-only
                  builds available recently. I always build my MacVim from source using
                  Python 2.5 and 10.5 SDK. I do also use a couple of extra patches in
                  my build (relative number, Lua, working on vimgdb). If there is
                  interest, I could put my builds up somewhere for others to use. My
                  main concern would be increasing the support load due to unofficial
                  builds floating around.

                  Jason

                  --~--~---------~--~----~------------~-------~--~----~
                  You received this message from the "vim_mac" maillist.
                  For more information, visit http://www.vim.org/maillist.php
                  -~----------~----~----~----~------~----~------~--~---
                • Matt Tolton
                  ... What about feature freezing the current 10.4 version and continuing development for 10.5 only? Then you could enable garbage collection which would be
                  Message 8 of 15 , Nov 3, 2008
                    > It seems this is the only way to go then. But...I wonder if it really
                    > is worth my time making two distributions just so a few users get to
                    > use Python 2.5 instead of 2.3. I think I need a better reason than
                    > that for something as drastic as distributing two different versions
                    > of the binary. Maybe I underestimate the number of users who would
                    > benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                    What about feature freezing the current 10.4 version and continuing
                    development for 10.5 only? Then you could enable garbage collection
                    which would be much more convenient for development -- this issue bugs
                    me especially because if someone wants to make a plugin, I'm pretty
                    sure they can't use garbage collection since MacVim doesn't.

                    --~--~---------~--~----~------------~-------~--~----~
                    You received this message from the "vim_mac" maillist.
                    For more information, visit http://www.vim.org/maillist.php
                    -~----------~----~----~----~------~----~------~--~---
                  • Nico Weber
                    ... That sounds reasonable. ... I don t use Tiger, but I don t think dropping Tiger already is a good idea (see e.g. http://update.omnigroup.com/). Nico
                    Message 9 of 15 , Nov 3, 2008
                      >> It seems this is the only way to go then. But...I wonder if it
                      >> really
                      >> is worth my time making two distributions just so a few users get to
                      >> use Python 2.5 instead of 2.3. I think I need a better reason than
                      >> that for something as drastic as distributing two different versions
                      >> of the binary. Maybe I underestimate the number of users who would
                      >> benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                      That sounds reasonable.

                      > What about feature freezing the current 10.4 version and continuing
                      > development for 10.5 only? Then you could enable garbage collection
                      > which would be much more convenient for development -- this issue bugs
                      > me especially because if someone wants to make a plugin, I'm pretty
                      > sure they can't use garbage collection since MacVim doesn't.

                      I don't use Tiger, but I don't think dropping Tiger already is a good
                      idea (see e.g. http://update.omnigroup.com/).

                      Nico

                      --~--~---------~--~----~------------~-------~--~----~
                      You received this message from the "vim_mac" maillist.
                      For more information, visit http://www.vim.org/maillist.php
                      -~----------~----~----~----~------~----~------~--~---
                    • Ben Schmidt
                      ... +1. Real Python-heads can always compile themselves. ... I agree, but I do use Tiger. 10.4.9. :-) Ben.
                      Message 10 of 15 , Nov 4, 2008
                        Nico Weber wrote:
                        >>> It seems this is the only way to go then. But...I wonder if it
                        >>> really
                        >>> is worth my time making two distributions just so a few users get to
                        >>> use Python 2.5 instead of 2.3. I think I need a better reason than
                        >>> that for something as drastic as distributing two different versions
                        >>> of the binary. Maybe I underestimate the number of users who would
                        >>> benefit from 2.5, but for now I'll just keep building with 2.3. :-/
                        >
                        > That sounds reasonable.

                        +1.

                        Real Python-heads can always compile themselves.

                        >> What about feature freezing the current 10.4 version and continuing
                        >> development for 10.5 only? Then you could enable garbage collection
                        >> which would be much more convenient for development -- this issue bugs
                        >> me especially because if someone wants to make a plugin, I'm pretty
                        >> sure they can't use garbage collection since MacVim doesn't.
                        >
                        > I don't use Tiger, but I don't think dropping Tiger already is a good
                        > idea (see e.g. http://update.omnigroup.com/).

                        I agree, but I do use Tiger. 10.4.9. :-)

                        Ben.




                        --~--~---------~--~----~------------~-------~--~----~
                        You received this message from the "vim_mac" maillist.
                        For more information, visit http://www.vim.org/maillist.php
                        -~----------~----~----~----~------~----~------~--~---
                      • Panos
                        I was pestering Bjorn about making python be dynamically loaded, but I went ahead a compiled from source. Easy as pie, plus now I have python 2.5 support :)
                        Message 11 of 15 , Nov 4, 2008
                          I was pestering Bjorn about making python be dynamically loaded, but I
                          went ahead a compiled from source. Easy as pie, plus now I have python
                          2.5 support :)

                          Thanks for making it that easy, although I still believe that people
                          that actually use python are way past 2.3 by now.

                          Still it's easier to have them compile it themselves, than to ask them
                          to move up a version of their OS, so I see why we're having this
                          issue.

                          It should be made easier to pick python version depending on the build
                          platform, hope we can figure this out.

                          On Nov 4, 12:17 pm, Ben Schmidt <mail_ben_schm...@...> wrote:
                          > Nico Weber wrote:
                          > >>> It seems this is the only way to go then.  But...I wonder if it  
                          > >>> really
                          > >>> is worth my time making two distributions just so a few users get to
                          > >>> use Python 2.5 instead of 2.3.  I think I need a better reason than
                          > >>> that for something as drastic as distributing two different versions
                          > >>> of the binary.  Maybe I underestimate the number of users who would
                          > >>> benefit from 2.5, but for now I'll just keep building with 2.3.  :-/
                          >
                          > > That sounds reasonable.
                          >
                          > +1.
                          >
                          > Real Python-heads can always compile themselves.
                          >
                          > >> What about feature freezing the current 10.4 version and continuing
                          > >> development for 10.5 only?  Then you could enable garbage collection
                          > >> which would be much more convenient for development -- this issue bugs
                          > >> me especially because if someone wants to make a plugin, I'm pretty
                          > >> sure they can't use garbage collection since MacVim doesn't.
                          >
                          > > I don't use Tiger, but I don't think dropping Tiger already is a good  
                          > > idea (see e.g.http://update.omnigroup.com/).
                          >
                          > I agree, but I do use Tiger. 10.4.9. :-)
                          >
                          > Ben.
                          --~--~---------~--~----~------------~-------~--~----~
                          You received this message from the "vim_mac" maillist.
                          For more information, visit http://www.vim.org/maillist.php
                          -~----------~----~----~----~------~----~------~--~---
                        • björn
                          ... Jason, feel free to make your own builds available -- I have no objections to this. (Sorry for taking so long to reply to your post!) Björn
                          Message 12 of 15 , Nov 7, 2008
                            2008/11/3 Jason Foreman <jason@...>:
                            >
                            > FWIW, I've been toying with the idea of making my own Leopard-only
                            > builds available recently. I always build my MacVim from source using
                            > Python 2.5 and 10.5 SDK. I do also use a couple of extra patches in
                            > my build (relative number, Lua, working on vimgdb). If there is
                            > interest, I could put my builds up somewhere for others to use. My
                            > main concern would be increasing the support load due to unofficial
                            > builds floating around.

                            Jason, feel free to make your own builds available -- I have no
                            objections to this.

                            (Sorry for taking so long to reply to your post!)

                            Björn

                            --~--~---------~--~----~------------~-------~--~----~
                            You received this message from the "vim_mac" maillist.
                            For more information, visit http://www.vim.org/maillist.php
                            -~----------~----~----~----~------~----~------~--~---
                          Your message has been successfully submitted and would be delivered to recipients shortly.