Loading ...
Sorry, an error occurred while loading the content.

Re: Test 86 fails with statis Python

Expand Messages
  • Yukihiro Nakadaira
    On Ubuntu 13.04, adding -fPIC to CFLAGS make it work. I don t know why. It seems that something is wrong with &_PyObject_NextNotImplemented extracted from
    Message 1 of 16 , Jun 19, 2013
    • 0 Attachment
      On Ubuntu 13.04, adding -fPIC to CFLAGS make it work.
      I don't know why.
      It seems that something is wrong with "&_PyObject_NextNotImplemented" extracted from PyIter_Check() macro.

      --
      Yukihiro Nakadaira - yukihiro.nakadaira@...

      --
      --
      You received this message from the "vim_dev" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
       
      ---
      You received this message because you are subscribed to the Google Groups "vim_dev" group.
      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
      For more options, visit https://groups.google.com/groups/opt_out.
       
       
    • Nikolay Pavlov
      ... Python. ... static ... does use ... non-debuging ... out ... case of ... my ... symbol ... failure ... I do not argue that we should do this. But I can t
      Message 2 of 16 , Jun 19, 2013
      • 0 Attachment


        On Jun 19, 2013 7:54 PM, "Bram Moolenaar" <Bram@...> wrote:
        >
        >
        > ZyX wrote:
        >
        > > > > On Jun 19, 2013 12:10 AM, "Bram Moolenaar" <Bram@...> wrote:
        > > > > >
        > > > > >
        > > > > > Test 86 passes without problems when using a dynamically loaded Python.
        > > > > > When using a static library I get the different output below.
        > > > > >
        > > > > > This is on Ubuntu 12.10 with Python 2.7.3.
        > > > > >
        > > > > > Test 87 has the same problem: OK with dynamic loading, fails with static
        > > > > > library.  See further down below.  Python 3.2.3.
        > > > >
        > > > > Unable to reproduce with neither debugging semi-static (it still does use
        > > > > ld-linux.so to link) python-2.3, -2.6 and -2.7 builds or with non-debuging
        > > > > semi-static python-2.7, though with the latter different error pops out
        > > > > which needs to be solved. There are also a few compiler warnings in case of
        > > > > semi-static build.
        > > >
        > > > Keep in mind that building with both python2 and python3 may
        > > > automatically switch do dynamic libraries.  At least that happens on my
        > > > system.
        > >
        > > Is not it the only option to build with both because of conflicting symbol
        > > names? Before testing I excluded python 3. And did not get *this* failure
        > > as I reported, though found some other one which is not in your report.
        >
        > I hope we can make the tests depend less on the specific Python version
        > and environment.  I understand it's not easy.  But ignoring test
        > failures because "that one always fails" will hide real problems.

        I do not argue that we should do this. But I can't fix things that work for me because I have no idea what is broken. Note that in your report + prefixes right lines though so I should focus not on deducing the reason of the difference, but find out why it shows NotImplementedError in my case (refer to the discussion of this issue happened previously). Likely this will fix the difference as well.

        > > By the way, is not it better to fail in place of switching to dynamic
        > > builds? I always compiled both versions with explicit =dynamic flags each
        > > time I wanted both pythons because I know it being the only option, but if
        > > I did not know this and requested semi-static ones and did get dynamic
        > > instead I would be surprised. The only case I would care about it is static
        > > build (fully static!) I once needed, but here I would be surprised very
        > > badly if I did not find python support due to such autoswitching.
        >
        > It's been like this for a long time.  In practice it appears to work.
        >
        > Of course it's not at all nice that :py3 does not work after :py, but
        > there is no good alternative.
        >
        > --
        > hundred-and-one symptoms of being an internet addict:
        > 252. You vote for foreign officials.
        >
        >  /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net   \\\
        > ///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
        > \\\  an exciting new programming language -- http://www.Zimbu.org        ///
        >  \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

        --
        --
        You received this message from the "vim_dev" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
         
        ---
        You received this message because you are subscribed to the Google Groups "vim_dev" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
        For more options, visit https://groups.google.com/groups/opt_out.
         
         
      • ZyX
        ... Indeed + lines stand for .ok file, not .failed. This reverts us back: I cannot fix what I cannot reproduce. Will check out whether I can install an Ubuntu
        Message 3 of 16 , Jun 19, 2013
        • 0 Attachment
          > I do not argue that we should do this. But I can't fix things that work for me because I have no idea what is broken. Note that in your report + prefixes right lines though so I should focus not on deducing the reason of the difference, but find out why it shows NotImplementedError in my case (refer to the discussion of this issue happened previously). Likely this will fix the difference as well.

          Indeed + lines stand for .ok file, not .failed. This reverts us back: I cannot fix what I cannot reproduce. Will check out whether I can install an Ubuntu python package by unpacking it to ~/{some-prefix}.

          --
          --
          You received this message from the "vim_dev" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php

          ---
          You received this message because you are subscribed to the Google Groups "vim_dev" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
          For more options, visit https://groups.google.com/groups/opt_out.
        • ZyX
          ... The interesting fact is that when I try to use the same expression (PyIter_Check) in the debugger (in the expanded form, of course) I get zero as expected.
          Message 4 of 16 , Jun 20, 2013
          • 0 Attachment
            On Wednesday, June 19, 2013 8:19:35 PM UTC+4, Yukihiro Nakadaira wrote:
            > On Ubuntu 13.04, adding -fPIC to CFLAGS make it work.
            > I don't know why.
            > It seems that something is wrong with "&_PyObject_NextNotImplemented" extracted from PyIter_Check() macro.
            >
            > --
            > Yukihiro Nakadaira - yukihiro....@...

            The interesting fact is that when I try to use the same expression (PyIter_Check) in the debugger (in the expanded form, of course) I get zero as expected. And the same bug if I put expanded form into the code in place of PyIter_Check (which is also expected though).

            Also with python-fixes branch PyIter_Check also matches FailingNumber class. This one does not even contain __iter__ method and thus should fail the very first check in PyIter_Check macros.

            I cannot say whose bug is this.

            --
            --
            You received this message from the "vim_dev" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php

            ---
            You received this message because you are subscribed to the Google Groups "vim_dev" group.
            To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
            For more options, visit https://groups.google.com/groups/opt_out.
          • ZyX
            ... From the `man gcc` it seems that `-fPIC` or at least `-fpic` is a must when compiling shared libraries. Should not the issue be then directed to Ubuntu
            Message 5 of 16 , Jun 20, 2013
            • 0 Attachment
              On Wednesday, June 19, 2013 8:19:35 PM UTC+4, Yukihiro Nakadaira wrote:
              > On Ubuntu 13.04, adding -fPIC to CFLAGS make it work.

              From the `man gcc` it seems that `-fPIC` or at least `-fpic` is a must when compiling shared libraries. Should not the issue be then directed to Ubuntu developers?

              --
              --
              You received this message from the "vim_dev" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php

              ---
              You received this message because you are subscribed to the Google Groups "vim_dev" group.
              To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
              For more options, visit https://groups.google.com/groups/opt_out.
            • Yukihiro Nakadaira
              ... I think so too. Maybe python requires explicit -fPIC flag for configure option. On Ubuntu 13.04 $ python -c import distutils.sysconfig;
              Message 6 of 16 , Jun 21, 2013
              • 0 Attachment
                On Fri, Jun 21, 2013 at 2:25 AM, ZyX <zyx.vim@...> wrote:
                On Wednesday, June 19, 2013 8:19:35 PM UTC+4, Yukihiro Nakadaira wrote:
                > On Ubuntu 13.04, adding -fPIC to CFLAGS make it work.

                From the `man gcc` it seems that `-fPIC` or at least `-fpic` is a must when compiling shared libraries. Should not the issue be then directed to Ubuntu developers?

                I think so too.
                Maybe python requires explicit -fPIC flag for configure option.

                On Ubuntu 13.04
                $ python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_config_var("CONFIG_ARGS"))'
                '--enable-shared' '--prefix=/usr' '--enable-ipv6' '--enable-unicode=ucs4' '--with-dbmliborder=bdb:gdbm' '--with-system-expat' '--with-system-ffi' '--with-fpectl' 'CC=x86_64-linux-gnu-gcc' 'CFLAGS=-D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security ' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro'

                On Fedora 18
                $ python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_config_var("CONFIG_ARGS"))'
                '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-ipv6' '--enable-shared' '--enable-unicode=ucs4' '--with-dbmliborder=gdbm:ndbm:bdb' '--with-system-expat' '--with-system-ffi' '--with-dtrace' '--with-tapset-install-dir=/usr/share/systemtap/tapset' '--with-valgrind' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CC=gcc' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4  -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv  ' 'LDFLAGS=-Wl,-z,relro   ' 'CPPFLAGS=-I/usr/lib64/libffi-3.0.10/include  '

                --
                Yukihiro Nakadaira - yukihiro.nakadaira@...

                --
                --
                You received this message from the "vim_dev" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php
                 
                ---
                You received this message because you are subscribed to the Google Groups "vim_dev" group.
                To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                For more options, visit https://groups.google.com/groups/opt_out.
                 
                 
              • Yukihiro Nakadaira
                On Sat, Jun 22, 2013 at 1:56 AM, Yukihiro Nakadaira
                Message 7 of 16 , Jun 21, 2013
                • 0 Attachment
                  On Sat, Jun 22, 2013 at 1:56 AM, Yukihiro Nakadaira <yukihiro.nakadaira@...> wrote:
                  Maybe python requires explicit -fPIC flag for configure option.

                  This assumption was wrong...

                  --
                  Yukihiro Nakadaira - yukihiro.nakadaira@...

                  --
                  --
                  You received this message from the "vim_dev" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php
                   
                  ---
                  You received this message because you are subscribed to the Google Groups "vim_dev" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                  For more options, visit https://groups.google.com/groups/opt_out.
                   
                   
                • Zulox4
                  ... Hello, For Linux Xubuntu 13, the ./configure --enable-shared for the last pyhton release 2.7.5 works, but the fPython 3.1.1 does not work :
                  Message 8 of 16 , Jun 21, 2013
                  • 0 Attachment
                    On Thursday, June 20, 2013 7:25:19 PM UTC+2, ZyX wrote:
                    > On Wednesday, June 19, 2013 8:19:35 PM UTC+4, Yukihiro Nakadaira wrote:
                    > > On Ubuntu 13.04, adding -fPIC to CFLAGS make it work.
                    >
                    > From the `man gcc` it seems that `-fPIC` or at least `-fpic` is a must when compiling shared libraries. Should not the issue be then directed to Ubuntu developers?

                    Hello,

                    For Linux Xubuntu 13, the ./configure --enable-shared for the last pyhton release 2.7.5 works, but the fPython 3.1.1 does not work :

                    http://stackoverflow.com/questions/1547310/python-3-1-1-with-enable-shared-will-not-build-any-extensions

                    Best regards!

                    --
                    --
                    You received this message from the "vim_dev" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php

                    ---
                    You received this message because you are subscribed to the Google Groups "vim_dev" group.
                    To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                    For more options, visit https://groups.google.com/groups/opt_out.
                  • Yukihiro Nakadaira
                    Cause of this problem is that ubuntu s python is built with -Bsymbolic-functions flag. With this flag, libpython and vim see different function address for
                    Message 9 of 16 , Jun 23, 2013
                    • 0 Attachment
                      Cause of this problem is that ubuntu's python is built with -Bsymbolic-functions flag.
                      With this flag, libpython and vim see different function address for same function.

                      Test:
                      $ cat foo.c
                      void *foo() { return &foo; }
                      $ cat main.c
                      #include <stdio.h>
                      void *foo();
                      int main() {
                          printf("foo=%p main=%p\n", foo(), &foo);
                          return 0;
                      }
                      $ cc -shared -fPIC -o libfoo.so foo.c -Wl,-Bsymbolic-functions
                      $ cc main.c -L. -lfoo
                      $ LD_LIBRARY_PATH=. ./a.out
                      foo=0x7eff0cafe770 main=0x4005e0

                      --
                      Yukihiro Nakadaira - yukihiro.nakadaira@...

                      --
                      --
                      You received this message from the "vim_dev" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php
                       
                      ---
                      You received this message because you are subscribed to the Google Groups "vim_dev" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                      For more options, visit https://groups.google.com/groups/opt_out.
                       
                       
                    • ZyX
                      ... Just filed a bug report: https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1196047. Let’s look, maybe we won’t have to do anything. As an
                      Message 10 of 16 , Jun 29, 2013
                      • 0 Attachment
                        On Sunday, June 23, 2013 3:25:52 PM UTC+4, Yukihiro Nakadaira wrote:
                        > Cause of this problem is that ubuntu's python is built with -Bsymbolic-functions flag.
                        > With this flag, libpython and vim see different function address for same function.
                        >
                        > Test:
                        > $ cat foo.c
                        >
                        > void *foo() { return &foo; }
                        > $ cat main.c
                        > #include <stdio.h>
                        > void *foo();
                        > int main() {
                        >     printf("foo=%p main=%p\n", foo(), &foo);
                        >     return 0;
                        > }
                        > $ cc -shared -fPIC -o libfoo.so foo.c -Wl,-Bsymbolic-functions
                        >
                        > $ cc main.c -L. -lfoo
                        > $ LD_LIBRARY_PATH=. ./a.out
                        > foo=0x7eff0cafe770 main=0x4005e0
                        >
                        >
                        > --
                        > Yukihiro Nakadaira - yukihiro....@...

                        Just filed a bug report: https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1196047. Let’s look, maybe we won’t have to do anything.

                        As an alternative I can suggest one of the following:

                        1. Detect -Wl,-Bsymbolic-functions flag in configure and fail or switch to =dynamic build. (I would really prefer failing.)
                        2. Add #if Py_UBUNTU_WORKAROUND macros guarding

                        #define _PyObject_NextNotImplemented dll__PyObject_NextNotImplemented
                        static iternextfunc dll__PyObject_NextNotImplemented;

                        at the top (in the #else branch of #if defined(DYNAMIC_PYTHON) || defined(PROTO)) and something like

                        PyObject *aobj;
                        PyRun_SimpleString("class A(object): pass");
                        if (PyErr_Occurred())
                        return -1;
                        if (!(aobj = PyRun_String("A", Py_eval_input, globals, globals)))
                        return -1;
                        dll__PyObject_NextNotImplemented = aobj->tp_iternext;
                        Py_DECREF(aobj);

                        after main python initialization. Macros should be added to CFLAGS by configure script if -Wl,\S*-Bsymbolic-functions is found in python LDFLAGS. This will workaround our problem as this way we get _PyObject_NextNotImplemented that is really used by python.

                        --
                        --
                        You received this message from the "vim_dev" maillist.
                        Do not top-post! Type your reply below the text you are replying to.
                        For more information, visit http://www.vim.org/maillist.php

                        ---
                        You received this message because you are subscribed to the Google Groups "vim_dev" group.
                        To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscribe@....
                        For more options, visit https://groups.google.com/groups/opt_out.
                      Your message has been successfully submitted and would be delivered to recipients shortly.