Shiv does not fully package namespaced packages
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 114
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I've run into a corner case when trying to package an application that uses tensorflow with shiv. You can reproduce the issue by creating a python 3.6 virtual environment with `tensorflow-cpu==2.6.0`, then running:
`shiv -c tensorboard -o brokenshiv tensorboard`
Tensorflow uses a few namespaced packages from google, which end up in a few locations:
- lib/python3.6/site-packages/google/auth
- lib/python3.6/site-packages/google/oauth
- lib64/python3.6/site-packages/google/protobuf
When shiv runs its packaging, it executes `pip install --target`, which triggers this linked issue:
https://github.com/pypa/pip/issues/10110
During the installation, there is a warning:
`WARNING: Target directory /tmp/tmpg5wic5v5/google already exists. Specify --upgrade to force replacement.`
In the unzipped shiv package, only the auth and oauth directories make it into site-packages under google; the protobuf package is missing.
In the pip issue, the maintainers suggest the correct behavior is to use `--prefix` during installation. Do you have a tip for how to solve this? Thanks!
Contributor guide
Research direction
Reproduce the packaging command with Python 3.6 and tensorflow-cpu==2.6.0, then inspect the unzipped shiv package's site-packages layout. Read the linked pip issue about --target and --prefix to understand the namespace-package failure. Done means the google/auth, google/oauth, and google/protobuf packages are all present in the generated archive.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100