indygreg / indygreg/PyOxidizer
Incorrect copied shared library name in filesystem relative resource policy
- Dominant language
- Rust
- Stars
- 6.2k
- Forks
- 256
- PR merge metrics
- No merged PRs in 30d
Description
When trying to use 0.7.0 to build a basic app with only numpy added, the new `prefer-in-memory-fallback-filesystem-relative` resource policy can be used. This almost works, but one of the shared libraries is copied into the incorrect location in the specified folder.
The `pyoxidizer.bzl` file is effectively the default generated file from the getting started guide, but with the following lines:
- `resources_policy='prefer-in-memory-fallback-filesystem-relative:lib',`
- `exe.add_filesystem_relative_python_resources('lib', dist.pip_install(['numpy']))`
When running the binary, then doing `import numpy`, the following (trimmed) error occurs:
`Original error was: libopenblasp-r0-ae94cfde.3.9.dev.so: cannot open shared object file: No such file or directory`
When looking at the file tree alongside the binary, there is the file at `build/x86_64-unknown-linux-gnu/debug/install/lib/numpy/libs/libopenblasp-r0-ae94cfde/3/9/dev.so`, which should instead be at `build/x86_64-unknown-linux-gnu/debug/install/lib/numpy.libs/libopenblasp-r0-ae94cfde.3.9.dev.so`
It appears `.` in the file name (`libopenblasp-r0-ae94cfde.3.9.dev.so`) and folder name (`numpy.libs`) have been converted to `/`. If you look in the numpy wheel, you can see the expected file tree structure. Having a vague dig through the PyOxidizer code, I can't see why this one file is getting mis-copied, particularly when there are three other shared libraries in that folder which are copied correctly. The only difference is the order of the version numbers around the `.so`. Is the shared lib path perhaps getting interpretted as a python module name?
If I manually move that file to the `numpy.libs` folder, numpy now imports correctly and works for the few things I've tested. If this can be resolved, numpy will effectively work out-of-the-box and resolve #65 .
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the issue with the generated pyoxidizer.bzl configuration, numpy, and the prefer-in-memory-fallback-filesystem-relative policy. Compare the NumPy wheel tree with the generated build/x86_64-unknown-linux-gnu/debug/install/lib paths, focusing on how the shared-library and numpy.libs names are copied. Done means the library remains at lib/numpy.libs/libopenblasp-r0-ae94cfde.3.9.dev.so and import numpy works without manual moves.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, rust
- Domain
- backend, build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100