linkedin / linkedin/shiv

Shiv does not fully package namespaced packages

Open
#191 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.9k
Forks
114
PR merge metrics
No merged PRs in 30d

Description

Hi, I've run into a corner case when trying to package an application that uses tensorflow with shiv. You can reproduce the issue by creating a python 3.6 virtual environment with `tensorflow-cpu==2.6.0`, then running:
`shiv -c tensorboard -o brokenshiv tensorboard`

Tensorflow uses a few namespaced packages from google, which end up in a few locations:
- lib/python3.6/site-packages/google/auth
- lib/python3.6/site-packages/google/oauth
- lib64/python3.6/site-packages/google/protobuf

When shiv runs its packaging, it executes `pip install --target`, which triggers this linked issue:
https://github.com/pypa/pip/issues/10110
During the installation, there is a warning:
`WARNING: Target directory /tmp/tmpg5wic5v5/google already exists. Specify --upgrade to force replacement.`

In the unzipped shiv package, only the auth and oauth directories make it into site-packages under google; the protobuf package is missing.

In the pip issue, the maintainers suggest the correct behavior is to use `--prefix` during installation. Do you have a tip for how to solve this? Thanks!

Contributor guide

Open the contributing guide

Research direction

Reproduce the packaging command with Python 3.6 and tensorflow-cpu==2.6.0, then inspect the unzipped shiv package's site-packages layout. Read the linked pip issue about --target and --prefix to understand the namespace-package failure. Done means the google/auth, google/oauth, and google/protobuf packages are all present in the generated archive.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.