Loading ...
Sorry, an error occurred while loading the content.

Re: Dynamically loading Python library

Expand Messages
  • Jason Foreman
    Hi Björn, ... I believe you must have installed the version of Python that your Vim distribution was built with. For standard builds I think this is 2.3.
    Message 1 of 15 , Nov 1, 2008
    • 0 Attachment
      Hi Björn,

      On Nov 1, 2008, at 11:57 AM, björn wrote:

      > How is this solved on Windows?

      I believe you must have installed the version of Python that your Vim
      distribution was built with. For standard builds I think this is
      2.3. Cream builds use 2.5 or maybe 2.4. I can verify this on Monday
      at work.

      > Please, even if nobody knows the answer to this question I urge you to

      > look into it and help me out, else I will be forced to continue
      > distributing MacVim with Python 2.3.


      I don't have an answer, but I'm willing to help if I can. I'm not
      100% sure there will be a solution short of having separate Leopard
      and Tiger builds, but it's worth trying.

      One thing that might work is to use "Py_InitModule3" instead of
      "Py_InitModule4" in if_python.c:2361. The former does not take an API
      version parameter, so that check won't be performed. Even the online
      Python docs [1] say that use of "Py_InitModule3" is preferred in most
      cases. I haven't got Tiger setup at the moment to try this out. I'll
      work on getting Tiger going this weekend so I can try your patch.

      Another thing to try would be downloading the Python 2.3 sources and
      using the headers from there when building, instead of the Python 2.5
      headers included with Leopard. This would ensure the
      PYTHON_API_VERSION matches. But I'm not sure if this would fix the
      problem or just invert it (no warn on Tiger, warn on Leopard).

      Jason

      [1] http://www.python.org/doc/2.5.2/api/allocating-objects.html
    • George V. Reilly
      2008/11/1 Jason Foreman : On Nov 1, 2008, at 11:57 AM, björn wrote: How is this solved on Windows? I believe you must have
      Message 2 of 15 , Nov 2, 2008
      • 0 Attachment
        2008/11/1 Jason Foreman <jason@...>:
        > On Nov 1, 2008, at 11:57 AM, björn wrote:
        >> How is this solved on Windows?
        >
        > I believe you must have installed the version of Python that your Vim
        > distribution was built with. For standard builds I think this is 2.3.
        > Cream builds use 2.5 or maybe 2.4. I can verify this on Monday at work.

        This is certainly correct for the Visual C++ build, Make_mvc.mak, and
        seems to be true of the MinGW build too.

        Try asking on http://mail.python.org/mailman/listinfo/capi-sig. That's
        where the experts on embedding Python hang out.

        If Python is dynamically loaded, then people running Python 2.3 on
        Tiger don't pay any cost until they use :python. I'm inclined to think
        that most people who want to use :python have Python 2.5 or 2.6
        installed.
        --
        /George V. Reilly george@...
        http://www.georgevreilly.com/blog http://blogs.cozi.com/tech

        --~--~---------~--~----~------------~-------~--~----~
        You received this message from the "vim_mac" maillist.
        For more information, visit http://www.vim.org/maillist.php
        -~----------~----~----~----~------~----~------~--~---
      • cfrees@imapmail.org
        ... In case it matters, Python 2.5 on Tiger (from python.org) does not include the dynamically linked shared library. This is so whether one uses the installer
        Message 3 of 15 , Nov 2, 2008
        • 0 Attachment
          On Sun 2nd Nov, 2008 at 17:42, George V. Reilly seems to have written:

          > 2008/11/1 Jason Foreman <jason@...>:
          >> On Nov 1, 2008, at 11:57 AM, bj?rn wrote:
          >>> How is this solved on Windows?
          >>
          >> I believe you must have installed the version of Python that your Vim
          >> distribution was built with. For standard builds I think this is 2.3.
          >> Cream builds use 2.5 or maybe 2.4. I can verify this on Monday at work.
          >
          > This is certainly correct for the Visual C++ build, Make_mvc.mak, and
          > seems to be true of the MinGW build too.
          >
          > Try asking on http://mail.python.org/mailman/listinfo/capi-sig. That's
          > where the experts on embedding Python hang out.
          >
          > If Python is dynamically loaded, then people running Python 2.3 on
          > Tiger don't pay any cost until they use :python. I'm inclined to think
          > that most people who want to use :python have Python 2.5 or 2.6
          > installed.

          In case it matters, Python 2.5 on Tiger (from python.org) does not
          include the dynamically linked shared library. This is so whether one
          uses the installer provided or whether one compiles the framework build
          from source and is true even if one explicitly requests that the
          library be built and installed. I don't have Leopard to compare, but I
          believe it does include the library in Python just as the Apple
          provided python 2.3 provides it on Tiger.

          As I say, I don't know if this matters or not.

          - cfr

          >

          --~--~---------~--~----~------------~-------~--~----~
          You received this message from the "vim_mac" maillist.
          For more information, visit http://www.vim.org/maillist.php
          -~----------~----~----~----~------~----~------~--~---
        • Ben Schmidt
          ... Hehe. I could have told you that and saved you the trouble. It s come up on one of the Vim mailing lists before. It s not solved on Windows, but IIRC,
          Message 4 of 15 , Nov 2, 2008
          • 0 Attachment
            > Now, I have managed to get Python to load dynamically and have tested
            > on Leopard/Intel and Tiger/PPC and on the former it loads 2.5, on the
            > latter 2.3. Yay! Or not. It turns out that there was an API change
            > between 2.3 and 2.5 and this is checked via a define
            > PYTHON_API_VERSION which is defined inside a Python header
            > (modsupport.h).

            Hehe. I could have told you that and saved you the trouble. It's come up
            on one of the Vim mailing lists before. It's not solved on Windows, but
            IIRC, gives even more drastic problems than it seems to on the Mac. :-\

            Unless you know exactly what the API changes are, you're pretty much
            stuffed. Any change in a structure Vim uses or the signature or return
            type of a function Vim calls, or a constant/macro definition Vim uses,
            and things will go wrong if the wrong version is dynamically loaded. In
            a sense, the warning is the least of your worries.

            You could ask the Python guys to try to find out exactly what the API
            changes are and whether they apply to Vim or not, but this could be
            difficult to ascertain--these things tend to be a bit of a haystack. On
            the other hand, it could be the change is very simple and irrelevant,
            and you can just somehow disable the warning--if this is a likely
            scenario, the Python guys almost certainly will have provided a way to
            disable it.

            If it can't be ascertained whether change is irrelevant, or it can be
            ascertained that it *is* relevant, it would be technically possible to
            make a Vim that worked with both API versions, but you'd have to
            basically duplicate if_python.c, change all the function names so there
            were two versions of each, compile one against the 2.3 API and another
            against the 2.5 API and then have a runtime check to decide which
            version of the functions to call. It would be quite a refactoring job,
            and I would say not worth the effort.

            One definite solution, though not ideal, but which is easy to set up for
            building, as well as easy for users to understand: have separate MacVim
            builds for Tiger and Leopard. Stacks of apps do it, and Mac users are
            fairly used to downloading the version appropriate to their OS version.

            Ben.




            --~--~---------~--~----~------------~-------~--~----~
            You received this message from the "vim_mac" maillist.
            For more information, visit http://www.vim.org/maillist.php
            -~----------~----~----~----~------~----~------~--~---
          • Ben Schmidt
            ... I m not so sure about that. I do use :python, but only indirectly, by using plugins that use it. As such, if it s good enough for the plugin, it s good
            Message 5 of 15 , Nov 2, 2008
            • 0 Attachment
              > If Python is dynamically loaded, then people running Python 2.3 on
              > Tiger don't pay any cost until they use :python. I'm inclined to think
              > that most people who want to use :python have Python 2.5 or 2.6
              > installed.

              I'm not so sure about that. I do use :python, but only indirectly, by
              using plugins that use it. As such, if it's good enough for the plugin,
              it's good enough for me, and I won't bother installing a later version
              than the one the OS provides unless something really requires it. I'm
              certainly not going to install one just for the love of Python or of
              being on the cutting edge. And to be honest, I wouldn't be sure that if
              I did install a later one, that would be the one MacVim would find first
              in the path. I suspect there are other users like me. In fact, I suspect
              more people use :python via plugins than use it directly in their own
              scripts or from the commandline.

              None of this actually affects me, as I compile MacVim from source
              myself, linking statically against whatever Python it finds. But I'm
              just pointing out that I expect it would affect others.

              Ben.




              --~--~---------~--~----~------------~-------~--~----~
              You received this message from the "vim_mac" maillist.
              For more information, visit http://www.vim.org/maillist.php
              -~----------~----~----~----~------~----~------~--~---
            • George V. Reilly
              2008/11/2 Ben Schmidt ... When I ported the Vim Python interface code to Win64, I looked around for Vim plugins to exercise it,
              Message 6 of 15 , Nov 2, 2008
              • 0 Attachment
                2008/11/2 Ben Schmidt <mail_ben_schmidt@...>
                > If Python is dynamically loaded, then people running Python 2.3 on
                > Tiger don't pay any cost until they use :python. I'm inclined to think
                > that most people who want to use :python have Python 2.5 or 2.6
                > installed.

                I'm not so sure about that. I do use :python, but only indirectly, by
                using plugins that use it.

                When I ported the Vim Python interface code to Win64, I looked around for Vim plugins to exercise it, and I had a hard time finding any. That was about a year ago. Maybe people have started using it more.

                Anyway, separate builds for Tiger and Leopard is probably the best way to go.
                --
                /George V. Reilly  george@...
                http://www.georgevreilly.com/blog  http://blogs.cozi.com/tech

                --~--~---------~--~----~------------~-------~--~----~
                You received this message from the "vim_mac" maillist.
                For more information, visit http://www.vim.org/maillist.php
                -~----------~----~----~----~------~----~------~--~---

              • Ben Schmidt
                ... I expect there are more plugins that use it on non-Windows systems. I think I have a couple installed that use it, but certainly the main one is
                Message 7 of 15 , Nov 2, 2008
                • 0 Attachment
                  George V. Reilly wrote:
                  > 2008/11/2 Ben Schmidt <mail_ben_schmidt@...
                  > <mailto:mail_ben_schmidt@...>>
                  >
                  > > If Python is dynamically loaded, then people running Python 2.3 on
                  > > Tiger don't pay any cost until they use :python. I'm inclined to
                  > think
                  > > that most people who want to use :python have Python 2.5 or 2.6
                  > > installed.
                  >
                  > I'm not so sure about that. I do use :python, but only indirectly, by
                  > using plugins that use it.
                  >
                  > When I ported the Vim Python interface code to Win64, I looked around
                  > for Vim plugins to exercise it, and I had a hard time finding any. That
                  > was about a year ago. Maybe people have started using it more.

                  I expect there are more plugins that use it on non-Windows systems.

                  I think I have a couple installed that use it, but certainly the main
                  one is

                  http://www.vim.org/scripts/script.php?script_id=1929

                  which I use from time to time to debug PHP. I first started using it
                  about a year ago too. It's the sort of thing that can make a big
                  difference to your coding experience, requires Python to be working, but
                  doesn't need the latest Python version. And certainly the user doesn't
                  want to be fiddling with any of the Python code or other Python-related
                  problems (like me, may not even know any Python). They already have
                  problems in their own code...

                  Grins,

                  Ben.




                  --~--~---------~--~----~------------~-------~--~----~
                  You received this message from the "vim_mac" maillist.
                  For more information, visit http://www.vim.org/maillist.php
                  -~----------~----~----~----~------~----~------~--~---
                • björn
                  ... I don t think it is a feasible solution...e.g. what happens the next time the API changes? No, I think loading Python dynamically is out of the question.
                  Message 8 of 15 , Nov 3, 2008
                  • 0 Attachment
                    2008/11/3 Ben Schmidt <mail_ben_schmidt@...>:
                    >
                    > You could ask the Python guys to try to find out exactly what the API
                    > changes are and whether they apply to Vim or not, but this could be
                    > difficult to ascertain--these things tend to be a bit of a haystack. On
                    > the other hand, it could be the change is very simple and irrelevant,
                    > and you can just somehow disable the warning--if this is a likely
                    > scenario, the Python guys almost certainly will have provided a way to
                    > disable it.

                    I don't think it is a feasible solution...e.g. what happens the next
                    time the API changes? No, I think loading Python dynamically is out
                    of the question. :(

                    > If it can't be ascertained whether change is irrelevant, or it can be
                    > ascertained that it *is* relevant, it would be technically possible to
                    > make a Vim that worked with both API versions, but you'd have to
                    > basically duplicate if_python.c, change all the function names so there
                    > were two versions of each, compile one against the 2.3 API and another
                    > against the 2.5 API and then have a runtime check to decide which
                    > version of the functions to call. It would be quite a refactoring job,
                    > and I would say not worth the effort.

                    Ugh, yeah, that's no good either.

                    > One definite solution, though not ideal, but which is easy to set up for
                    > building, as well as easy for users to understand: have separate MacVim
                    > builds for Tiger and Leopard. Stacks of apps do it, and Mac users are
                    > fairly used to downloading the version appropriate to their OS version.

                    It seems this is the only way to go then. But...I wonder if it really
                    is worth my time making two distributions just so a few users get to
                    use Python 2.5 instead of 2.3. I think I need a better reason than
                    that for something as drastic as distributing two different versions
                    of the binary. Maybe I underestimate the number of users who would
                    benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                    Thanks to everybody for the feedback,
                    Björn

                    --~--~---------~--~----~------------~-------~--~----~
                    You received this message from the "vim_mac" maillist.
                    For more information, visit http://www.vim.org/maillist.php
                    -~----------~----~----~----~------~----~------~--~---
                  • Jason Foreman
                    On Mon, Nov 3, 2008 at 11:55 AM, björn wrote: 2008/11/3 Ben Schmidt : have separate MacVim
                    Message 9 of 15 , Nov 3, 2008
                    • 0 Attachment
                      On Mon, Nov 3, 2008 at 11:55 AM, björn <bjorn.winckler@...> wrote:
                      > 2008/11/3 Ben Schmidt <mail_ben_schmidt@...>:
                      >> have separate MacVim builds for Tiger and Leopard.
                      >
                      > It seems this is the only way to go then. But...I wonder if it really
                      > is worth my time making two distributions just so a few users get to
                      > use Python 2.5 instead of 2.3. I think I need a better reason than
                      > that for something as drastic as distributing two different versions
                      > of the binary. Maybe I underestimate the number of users who would
                      > benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                      FWIW, I've been toying with the idea of making my own Leopard-only
                      builds available recently. I always build my MacVim from source using
                      Python 2.5 and 10.5 SDK. I do also use a couple of extra patches in
                      my build (relative number, Lua, working on vimgdb). If there is
                      interest, I could put my builds up somewhere for others to use. My
                      main concern would be increasing the support load due to unofficial
                      builds floating around.

                      Jason

                      --~--~---------~--~----~------------~-------~--~----~
                      You received this message from the "vim_mac" maillist.
                      For more information, visit http://www.vim.org/maillist.php
                      -~----------~----~----~----~------~----~------~--~---
                    • Matt Tolton
                      ... What about feature freezing the current 10.4 version and continuing development for 10.5 only? Then you could enable garbage collection which would be
                      Message 10 of 15 , Nov 3, 2008
                      • 0 Attachment
                        > It seems this is the only way to go then. But...I wonder if it really
                        > is worth my time making two distributions just so a few users get to
                        > use Python 2.5 instead of 2.3. I think I need a better reason than
                        > that for something as drastic as distributing two different versions
                        > of the binary. Maybe I underestimate the number of users who would
                        > benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                        What about feature freezing the current 10.4 version and continuing
                        development for 10.5 only? Then you could enable garbage collection
                        which would be much more convenient for development -- this issue bugs
                        me especially because if someone wants to make a plugin, I'm pretty
                        sure they can't use garbage collection since MacVim doesn't.

                        --~--~---------~--~----~------------~-------~--~----~
                        You received this message from the "vim_mac" maillist.
                        For more information, visit http://www.vim.org/maillist.php
                        -~----------~----~----~----~------~----~------~--~---
                      • Nico Weber
                        ... That sounds reasonable. ... I don t use Tiger, but I don t think dropping Tiger already is a good idea (see e.g. http://update.omnigroup.com/). Nico
                        Message 11 of 15 , Nov 3, 2008
                        • 0 Attachment
                          >> It seems this is the only way to go then. But...I wonder if it
                          >> really
                          >> is worth my time making two distributions just so a few users get to
                          >> use Python 2.5 instead of 2.3. I think I need a better reason than
                          >> that for something as drastic as distributing two different versions
                          >> of the binary. Maybe I underestimate the number of users who would
                          >> benefit from 2.5, but for now I'll just keep building with 2.3. :-/

                          That sounds reasonable.

                          > What about feature freezing the current 10.4 version and continuing
                          > development for 10.5 only? Then you could enable garbage collection
                          > which would be much more convenient for development -- this issue bugs
                          > me especially because if someone wants to make a plugin, I'm pretty
                          > sure they can't use garbage collection since MacVim doesn't.

                          I don't use Tiger, but I don't think dropping Tiger already is a good
                          idea (see e.g. http://update.omnigroup.com/).

                          Nico

                          --~--~---------~--~----~------------~-------~--~----~
                          You received this message from the "vim_mac" maillist.
                          For more information, visit http://www.vim.org/maillist.php
                          -~----------~----~----~----~------~----~------~--~---
                        • Ben Schmidt
                          ... +1. Real Python-heads can always compile themselves. ... I agree, but I do use Tiger. 10.4.9. :-) Ben.
                          Message 12 of 15 , Nov 4, 2008
                          • 0 Attachment
                            Nico Weber wrote:
                            >>> It seems this is the only way to go then. But...I wonder if it
                            >>> really
                            >>> is worth my time making two distributions just so a few users get to
                            >>> use Python 2.5 instead of 2.3. I think I need a better reason than
                            >>> that for something as drastic as distributing two different versions
                            >>> of the binary. Maybe I underestimate the number of users who would
                            >>> benefit from 2.5, but for now I'll just keep building with 2.3. :-/
                            >
                            > That sounds reasonable.

                            +1.

                            Real Python-heads can always compile themselves.

                            >> What about feature freezing the current 10.4 version and continuing
                            >> development for 10.5 only? Then you could enable garbage collection
                            >> which would be much more convenient for development -- this issue bugs
                            >> me especially because if someone wants to make a plugin, I'm pretty
                            >> sure they can't use garbage collection since MacVim doesn't.
                            >
                            > I don't use Tiger, but I don't think dropping Tiger already is a good
                            > idea (see e.g. http://update.omnigroup.com/).

                            I agree, but I do use Tiger. 10.4.9. :-)

                            Ben.




                            --~--~---------~--~----~------------~-------~--~----~
                            You received this message from the "vim_mac" maillist.
                            For more information, visit http://www.vim.org/maillist.php
                            -~----------~----~----~----~------~----~------~--~---
                          • Panos
                            I was pestering Bjorn about making python be dynamically loaded, but I went ahead a compiled from source. Easy as pie, plus now I have python 2.5 support :)
                            Message 13 of 15 , Nov 4, 2008
                            • 0 Attachment
                              I was pestering Bjorn about making python be dynamically loaded, but I
                              went ahead a compiled from source. Easy as pie, plus now I have python
                              2.5 support :)

                              Thanks for making it that easy, although I still believe that people
                              that actually use python are way past 2.3 by now.

                              Still it's easier to have them compile it themselves, than to ask them
                              to move up a version of their OS, so I see why we're having this
                              issue.

                              It should be made easier to pick python version depending on the build
                              platform, hope we can figure this out.

                              On Nov 4, 12:17 pm, Ben Schmidt <mail_ben_schm...@...> wrote:
                              > Nico Weber wrote:
                              > >>> It seems this is the only way to go then.  But...I wonder if it  
                              > >>> really
                              > >>> is worth my time making two distributions just so a few users get to
                              > >>> use Python 2.5 instead of 2.3.  I think I need a better reason than
                              > >>> that for something as drastic as distributing two different versions
                              > >>> of the binary.  Maybe I underestimate the number of users who would
                              > >>> benefit from 2.5, but for now I'll just keep building with 2.3.  :-/
                              >
                              > > That sounds reasonable.
                              >
                              > +1.
                              >
                              > Real Python-heads can always compile themselves.
                              >
                              > >> What about feature freezing the current 10.4 version and continuing
                              > >> development for 10.5 only?  Then you could enable garbage collection
                              > >> which would be much more convenient for development -- this issue bugs
                              > >> me especially because if someone wants to make a plugin, I'm pretty
                              > >> sure they can't use garbage collection since MacVim doesn't.
                              >
                              > > I don't use Tiger, but I don't think dropping Tiger already is a good  
                              > > idea (see e.g.http://update.omnigroup.com/).
                              >
                              > I agree, but I do use Tiger. 10.4.9. :-)
                              >
                              > Ben.
                              --~--~---------~--~----~------------~-------~--~----~
                              You received this message from the "vim_mac" maillist.
                              For more information, visit http://www.vim.org/maillist.php
                              -~----------~----~----~----~------~----~------~--~---
                            • björn
                              ... Jason, feel free to make your own builds available -- I have no objections to this. (Sorry for taking so long to reply to your post!) Björn
                              Message 14 of 15 , Nov 7, 2008
                              • 0 Attachment
                                2008/11/3 Jason Foreman <jason@...>:
                                >
                                > FWIW, I've been toying with the idea of making my own Leopard-only
                                > builds available recently. I always build my MacVim from source using
                                > Python 2.5 and 10.5 SDK. I do also use a couple of extra patches in
                                > my build (relative number, Lua, working on vimgdb). If there is
                                > interest, I could put my builds up somewhere for others to use. My
                                > main concern would be increasing the support load due to unofficial
                                > builds floating around.

                                Jason, feel free to make your own builds available -- I have no
                                objections to this.

                                (Sorry for taking so long to reply to your post!)

                                Björn

                                --~--~---------~--~----~------------~-------~--~----~
                                You received this message from the "vim_mac" maillist.
                                For more information, visit http://www.vim.org/maillist.php
                                -~----------~----~----~----~------~----~------~--~---
                              Your message has been successfully submitted and would be delivered to recipients shortly.