indygreg / indygreg/PyOxidizer
Shared lib builder drops package directory
- Dominant language
- Rust
- Stars
- 6.2k
- Forks
- 256
- PR merge metrics
- No merged PRs in 30d
Description
Shared libs that are in packages that also contain python code are stored in `site-packages/foo/bar.so`
PyOxidizer is dropping the `foo` package name, creating conflicts.
A very prominent example is `markupsafe`, `simplejson` and `logbook` all have a `_speedups.so`
```py
>>> import simplejson
import simplejson as json
File "simplejson", line 115, in
File "simplejson.encoder", line 15, in
File "simplejson.encoder", line 12, in _import_speedups
AttributeError: module 'markupsafe._speedups' has no attribute 'encode_basestring_ascii'
```
`tornado` and `websockets` and `fastparquet` all have a `speedups.so`
`cytoolz` and `h5py` and `espressomd` all have `utils.so`, and further nested are `zmq/backend/cython/utils.so` and `yt/analysis_modules/photon_simulator/utils.so`
`dulwich` and `h5py` have `_objects.so`
`xxhash` and `lmdb` have `cpython.so`
`bsdiff4` and `espeak` have `core.so`
`astropy` and `sunpy` have `_compiler.so`
`gevent` and `kivy` both have `_event.so`
PyQt4 and PyQt5 have an almost identical set of built modules under their respective directories.
Cryptodome and Crypto also share a set of similar names.
Third level objects `sklearn/manifold/_utils.so` and `sklearn/tree/_utils.so` probably also have this problem, breaking sklearn even if only it is installed.
Third level objects `gssapi/raw/misc.so` and `samba/dcerpc/misc.so` likely also conflict.
Third level `astropy/stats/_stats.so` and `scipy/stats/_stats.so`
As noted at https://github.com/indygreg/PyOxidizer/issues/167 , brotli and brotlipy both expect `from brotli import _brotli` to import their .so , but that example is a bit messy as the two libraries conflict anyway.
And that is just what I can find in my site-packages :P Obviously there will be many more conflicts I am unaware of.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the collision with packages such as markupsafe, simplejson, and logbook, then trace the shared lib builder responsible for paths under site-packages. Verify that nested package directories are preserved so same-named .so files remain importable without collisions; the issue names no test or source file, so repository-wide search is required.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100