python-poetry / python-poetry/poetry

"Managing environments" confusing documentation about honoring current virtual environment

Open
#8,247 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/docs status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I have searched the issues of this repo and believe that this is not a duplicate.

Issue

The Poetry docs on "Managing environments" opens with this simple statement:

... it [Poetry] will first check if it’s currently running inside a virtual environment. If it is, it will use it directly without creating a new one.

This doesn't seem to be entirely true, as poetry env use seems to take precedence over the current virtual environment.

Proof

Set up new directory with fresh venv:

$ mkdir poetry-venv-confusion-demo
$ cd poetry-venv-confusion-demo

Init poetry + see that it does not see a venv:

$ poetry init -n
$ poetry env info

Virtualenv
Python:         3.10.12
Implementation: CPython
Path:           NA
Executable:     NA
...

Do something that forces poetry to create a venv:

$ poetry install
Creating virtualenv poetry-venv-confusion-demo-TAVwgqF2-py3.10 in /home/jeremy/.cache/pypoetry/virtualenvs
...

Now poetry sees that venv it just created:

$ poetry env info

Virtualenv
Python:         3.10.12
Implementation: CPython
Path:           /home/jeremy/.cache/pypoetry/virtualenvs/poetry-venv-confusion-demo-TAVwgqF2-py3.10
Executable:     /home/jeremy/.cache/pypoetry/virtualenvs/poetry-venv-confusion-demo-TAVwgqF2-py3.10/bin/python
Valid:          True

Create + activate venv + proof it's activated:

$ python -m venv .my-venv  # poetry has special handling of the .venv directory, so choosing something else here
$ source .my-venv/bin/activate
$ which python
/home/jeremy/tmp/poetry-venv-confusion-demo/.my-venv/bin/python
$ echo $VIRTUAL_ENV
/home/jeremy/tmp/poetry-venv-confusion-demo/.my-venv

See that poetry honors the current active venv:

$ poetry env info

Virtualenv
Python:         3.10.12
Implementation: CPython
Path:           /home/jeremy/tmp/poetry-venv-confusion-demo/.my-venv
Executable:     /home/jeremy/tmp/poetry-venv-confusion-demo/.my-venv/bin/python
Valid:          True
...

So far so good. As promised, poetry is preferring the active venv (in .my-venv) over the one it created in ~/.cache//pypoetry/virtualenvs earlier.

Here's the rub. If I run poetry env use, then poetry stops honoring my active venv:

$ poetry env use python3.10
Using virtualenv: /home/jeremy/.cache/pypoetry/virtualenvs/poetry-venv-confusion-demo-TAVwgqF2-py3.10

$ poetry env info

Virtualenv
Python:         3.10.12
Implementation: CPython
Path:           /home/jeremy/.cache/pypoetry/virtualenvs/poetry-venv-confusion-demo-TAVwgqF2-py3.10
Executable:     /home/jeremy/.cache/pypoetry/virtualenvs/poetry-venv-confusion-demo-TAVwgqF2-py3.10/bin/python
Valid:          True

Here's the relevant code: https://github.com/python-poetry/poetry/blob/1.5.1/src/poetry/utils/env.py#L680-L684. Running poetry env use creates a ~/.cache/pypoetry/virtualenvs/envs.toml (the same as the TOMLFile(venv_path / self.ENVS_FILE) you see in the code) that takes precedence over the fact that we're currently in a venv.

I can see arguments for poetry choosing to behave this way. It's easy to imagine users being surprised if running poetry env use python3.10 had no effect because they're actually already in a venv. That said, the current behavior is extremely confusing for people who want to bring their own venv management. If the user ever ran poetry env use, they're going to be pretty surprised to find that poetry has no interest in the venv that they've so carefully activated.

More context: I am on a team of ~40 engineers at a python shop. We use direnv to manage venvs for our projects, and we really just want poetry to be a tool for managing python deps, we do not want it to manage venvs. I ended up in a bit of a marathon debugging session earlier this week because poetry wasn't installing deps into a developer's venv, and it took us forever to discover that this was because he had a envs.toml file in his poetry cache dir (presumably because he ran poetry env use at some distant point in the past).

Would y'all be open to changing poetry to make these sorts of environment misconfigs less likely to happen or less confusing? Some ideas:

  • Have poetry complain (perhaps even abort) with a clear error/warning message if it finds that there's both a current venv and an entry in the envs.toml file for the current directory.
  • Add a config setting to put poetry in a "do not do any venv management" mode
  • Or, if neither of the above are palatable, change the "Managing environments" docs to explain that poetry prefers poetry use environments over a currently active venv.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the “Managing environments” documentation section linked in the issue, then compare its wording with the environment-selection code at src/poetry/utils/env.py#L680-L684. Reproduce the documented and poetry env use scenarios to confirm the precedence behavior. Done would require an agreed resolution—such as clarified documentation or a specified behavior change—plus corresponding tests if behavior changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.