Issue 10126: test_distutils failure with --enable-shared

This happens on Python 3.1:

/usr/bin/ld: cannot find -lpython3.1
collect2: ld returned 1 exit status
test test_distutils failed -- Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/unixccompiler.py", line 254, in link
    self.spawn(linker + ld_args)
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/spawn.py", line 34, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/spawn.py", line 138, in _spawn_posix
    % (cmd[0], exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/tests/test_build_ext.py", line 321, in test_get_outputs
    cmd.run()
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/command/build_ext.py", line 347, in run
    self.build_extensions()
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/command/build_ext.py", line 456, in build_extensions
    self.build_extension(ext)
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/command/build_ext.py", line 543, in build_extension
    target_lang=language)
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/ccompiler.py", line 719, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/srv/buildbot/buildarea/3.1.bolen-ubuntu/build/Lib/distutils/unixccompiler.py", line 256, in link
    raise LinkError(msg)
distutils.errors.LinkError: command 'gcc' failed with exit status 1

(from http://www.python.org/dev/buildbot/builders/x86%20Ubuntu%20Shared%203.1/builds/1132/steps/test/logs/stdio )

Since it is on one of the stable buildbots, it might be considered release-blocking.

Similar failure on 2.7:

/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
test test_distutils failed -- Traceback (most recent call last):
  File "/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Lib/distutils/tests/test_build_ext.py", line 269, in test_get_outputs
    cmd.run()
  File "/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Lib/distutils/command/build_ext.py", line 449, in build_extensions
    self.build_extension(ext)
  File "/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Lib/distutils/command/build_ext.py", line 531, in build_extension
    target_lang=language)
  File "/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Lib/distutils/ccompiler.py", line 741, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Lib/distutils/unixccompiler.py", line 258, in link
    raise LinkError, msg
LinkError: command 'gcc' failed with exit status 1

I fixed this in py3k branch, so you just need to backport that.  I can do if you can't get to it in the next day or so.

Le samedi 16 octobre 2010 à 20:43 +0000, Barry A. Warsaw a écrit :
> Barry A. Warsaw <barry@python.org> added the comment:
> 
> I fixed this in py3k branch, so you just need to backport that.  I can
> do if you can't get to it in the next day or so.

I don't know how to do it so, yes, it would be better if you do :)
Thanks for caring!

If I understand correctly, only the changes you made to test_build_ext.py have to be backported.  I can do it if you want, just assign to me, or else do it and I’ll forward port to distutils2.

Note that I don’t fully understand the change, but I trust you that it is indeed the test that was faulty and not the code itself.

Hi Eric, yes, that's basically what I did.  See _fix_command(), which *should* be the only thing that needs to be backported.  I do believe it's a faulty test and hopefully the comment explains why it's necessary.

Thanks for giving the backport a shot!

Hi Éric, what's the status on the backport?

Done in r85779 and r85780, please test.

The backport fixed test_build_ext (the method) but not test_get_outputs.  Barry and I are on it.

It's possible the proposed patch for 2.7 should be used in python 3 also.  Or as __ap__ suggestions in irc, os.path.dirname(sys.executable)

Fixed in 2.7 by r85784

Applied on py3k in r85784.  I had no bug with 3.1, so I left things alone.

That was actually r85786.

You broke many 2.7, 3.1 and 3.x buildbots.

I believe all known failures in 2.7, 3.1, and 3.2 both with and without --enable-shared are now fixed.  Let's see what the buildbots say.

distutils2 has been converted to 3.x and will be merged into 3.3 under the name packaging shortly.  The fix has not been ported; I applied the patch for build_ext and its test but didn’t succeed.

I am trying to fix this by backporting the distutils fix, but --enabled-share does not work for me ?!

$ make distclean && ./configure --enable-shared --with-pydebug && make
$ ./python
libpython3.3dm.so.1.0: cannot open shared object file: No such file or directory

The so file is present in the srcdir though.

Investigating...

Tarek, you must use LD_LIBRARY_PATH.

New changeset 3d822e1ca930 by Tarek Ziade in branch 'default':
Issue 10126 - backported distutils fix for the enable-share mode into packaging
http://hg.python.org/cpython/rev/3d822e1ca930

Fixed in packaging. It seems that all fronts are OK, so closing this

messages: + msg136564

messages: + msg136247
versions: + Python 3.3

messages: + msg119343

messages: + msg119328
resolution: fixed
stage: needs patch -> resolved

assignee: tarek
components: + Distutils2