Runtime libraries conflict
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 1.4k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 1
Description
This is a quite general issue and I'm not sure that it relates to the setuptools directly, but there should be some starting point, and I don't know what is the best place to raise this question.
Anyway, here's my environment: FreeBSD 11/Python 3.5/setuptools 32.3.1
There are some math libraries that have been installed from ports: blas/openblas/lapack/atlas. All these libs have been built with gcc and they depend on its runtime libs. It looks like this:
# ldd /usr/local/lib/liblapack.so.4
/usr/local/lib/liblapack.so.4:
libgfortran.so.3 => /usr/local/lib/gcc49/libgfortran.so.3 (0x801a00000)
libm.so.5 => /lib/libm.so.5 (0x801e65000)
libgcc_s.so.1 => /usr/local/lib/gcc49/libgcc_s.so.1 (0x80208f000)
libquadmath.so.0 => /usr/local/lib/gcc49/libquadmath.so.0 (0x8022a6000)
# ldd /usr/local/lib/gcc49/libgfortran.so.3
/usr/local/lib/gcc49/libgfortran.so.3:
libquadmath.so.0 => /usr/local/lib/gcc49/libquadmath.so.0 (0x801465000)
libm.so.5 => /lib/libm.so.5 (0x8016b3000)
libgcc_s.so.1 => /usr/local/lib/gcc49/libgcc_s.so.1 (0x8018dd000)
libc.so.7 => /lib/libc.so.7 (0x800823000)
Pay attention for the path of the libgcc_s.so.1 library.
Then, there are some python packages have been installed with pip, like: scipy, scikit-learn and so on. And here is an interesting point. FreeBSD uses Clang as a system compiler, and setuptools/distutils uses the system compiler to build some system-dependent parts of the packages. So, these packages have been build with different compiler and they depend on different set of runtime libs. At the same time these packages depend on the libs from ports. It looks like this:
# ldd /usr/local/lib/python3.5/site-packages/scipy/sparse/linalg/isolve/_iterative.so
/usr/local/lib/python3.5/site-packages/scipy/sparse/linalg/isolve/_iterative.so:
liblapack.so.4 => /usr/local/lib/liblapack.so.4 (0x801400000)
libcblas.so.2 => /usr/local/lib/libcblas.so.2 (0x801c34000)
libblas.so.2 => /usr/local/lib/libblas.so.2 (0x801e51000)
libpython3.5m.so.1.0 => /usr/local/lib/libpython3.5m.so.1.0 (0x802200000)
libgfortran.so.3 => /usr/local/lib/gcc49/libgfortran.so.3 (0x802800000)
libm.so.5 => /lib/libm.so.5 (0x802c65000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x802e8f000)
libquadmath.so.0 => /usr/local/lib/gcc49/libquadmath.so.0 (0x80309d000)
libc.so.7 => /lib/libc.so.7 (0x800823000)
libatlas.so.2 => /usr/local/lib/libatlas.so.2 (0x803400000)
libthr.so.3 => /lib/libthr.so.3 (0x803977000)
libintl.so.8 => /usr/local/lib/libintl.so.8 (0x803b9e000)
libutil.so.9 => /lib/libutil.so.9 (0x803da9000)
As a result, the runtime libraries conflict happens when one of the packages is imported:
Traceback (most recent call last):
File "./***py", line 13, in <module>
from sklearn import preprocessing
File "/usr/local/lib/python3.5/site-packages/sklearn/__init__.py", line 57, in <module>
from .base import clone
File "/usr/local/lib/python3.5/site-packages/sklearn/base.py", line 12, in <module>
from .utils.fixes import signature
File "/usr/local/lib/python3.5/site-packages/sklearn/utils/__init__.py", line 11, in <module>
from .validation import (as_float_array,
File "/usr/local/lib/python3.5/site-packages/sklearn/utils/validation.py", line 18, in <module>
from ..utils.fixes import signature
File "/usr/local/lib/python3.5/site-packages/sklearn/utils/fixes.py", line 291, in <module>
from scipy.sparse.linalg import lsqr as sparse_lsqr
File "/usr/local/lib/python3.5/site-packages/scipy/sparse/linalg/__init__.py", line 112, in <module>
from .isolve import *
File "/usr/local/lib/python3.5/site-packages/scipy/sparse/linalg/isolve/__init__.py", line 6, in <module>
from .iterative import *
File "/usr/local/lib/python3.5/site-packages/scipy/sparse/linalg/isolve/iterative.py", line 7, in <module>
from . import _iterative
ImportError: /lib/libgcc_s.so.1: version GCC_4.6.0 required by /usr/local/lib/gcc49/libgfortran.so.3 not found
I've heard there are some workarounds for this issue, but I think there should be more elegant solution.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the scipy/sklearn import on FreeBSD 11 with Python 3.5 and inspect setuptools/distutils compiler and runtime-library handling alongside the ldd output. Define done as an agreed, tested way to prevent or resolve the mixed libgcc_s/libgfortran conflict; the issue does not name a source file or test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100