python / python/cpython

Virtual environments created with `venv --system-site-packages` can't access binaries from pre-installed packages, neither re-install those packages

Open
#118,388 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

I wanted to "pre-install" in my "source" python installation some packages I often use, so that those packages are readily available in the virtual environments created from this "source", without wasting space and time re-installing them every time I create a new venv.

I thought the --system-site-packages would do just that, however, this "doesn't work" for binaries installed by python packages (for instance, ipython, but I can think of other cases like pylint, flake8 or cookieninja):

I create the venv (using a locally compiled 3.11 version):

bli@latimeria:~/tmp$ ~/python3.11/bin/python3 -m venv --system-site-packages .venv
bli@latimeria:~/tmp$ source .venv/bin/activate

However, the bin directory of the venv does not contain ipython:

(.venv) bli@latimeria:~/tmp$ ls .venv/bin/ | grep ipython

But ipython is installed in the source python 3.11:

(.venv) bli@latimeria:~/tmp$ ls ~/python3.11/bin/ | grep ipython
ipython
ipython3

This is however not in the PATH when in the venv (in this particular case, calling ipython starts one installed with python 3.6 in ${HOME}/bin, whereas if I try with a package not installed anywhere else than in the python 3.11 install, like cookieninja, the command is not found).

This even prevents the installation of ipython in the venv:

(.venv) bli@latimeria:~/tmp$ .venv/bin/pip3 install ipython
Requirement already satisfied: ipython in /home/bli/python3.11/lib/python3.11/site-packages (8.24.0)
[...]
(.venv) bli@latimeria:~/tmp$ .venv/bin/pip3 install --upgrade ipython
Requirement already satisfied: ipython in /home/bli/python3.11/lib/python3.11/site-packages (8.24.0)
[...]
CPython versions tested on:

3.11

Operating systems tested on:

Linux

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with python -m venv --system-site-packages, activation, and packages such as ipython or cookieninja installed in the source Python. Compare the venv PATH and pip install or upgrade behavior; done means the expected access to pre-installed package binaries and package installation behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.