python-poetry / python-poetry/poetry

Activated-virtualenv-detection (including in tox) fails if there's also an active, poetry-managed, out-of-project env

Open
#2,888 13 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/venv kind/bug status/needs-consensus
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I am on the latest Poetry version.

  • I have searched the issues of this repo and believe that this is not a duplicate.

  • OS version and name: MacOS 10.15.6 10.15.7

  • Poetry version: 1.1.0b2 up to current 1.1.6

  • Link of a Gist with the contents of your pyproject.toml file: n/a

Issue

Detection of an active virtualenv doesn't work if there's an out-of-project poetry-managed env "active" (i.e. listed as "Activated" in poetry env list).

-----------> poetry env list
-----------> poetry env info -p
-----------> source ~/.pyenv/versions/external_venv/bin/activate    # Activate an external venv

(external_venv) -----------> poetry env info -p
/Users/mikenerone/.pyenv/versions/3.8.5/envs/external_venv  # So far, so good

(external_venv) -----------> poetry env use 3.8    # Create a poetry-managed venv
Creating virtualenv superapp-HVMyRI9d-py3.8 in /Users/mikenerone/Library/Caches/pypoetry/virtualenvs
Using virtualenv: /Users/mikenerone/Library/Caches/pypoetry/virtualenvs/superapp-HVMyRI9d-py3.8  # First sign of trouble

(external_venv) -----------> poetry env info -p
/Users/mikenerone/Library/Caches/pypoetry/virtualenvs/superapp-HVMyRI9d-py3.8  # BAD: My activated venv is no longer detected

(external_venv) -----------> poetry env remove 3.8    # Remove the poetry-managed venv
Deleted virtualenv: /Users/mikenerone/Library/Caches/pypoetry/virtualenvs/superapp-HVMyRI9d-py3.8

(external_venv) -----------> poetry env info -p
/Users/mikenerone/.pyenv/versions/3.8.5/envs/external_venv  # Removing the poetry-managed venv restored the active-venv detection

This breaks any tox functionality needing poetry to perform any operations on the virtualenv, such as poetry install (unless you work around the issue by doing poetry env remove <every-poetry-managed-venv> before executing tox), because Poetry works with its own virtualenv instead of updating the one that tox built, and in which it will be running the other configured commands. Even worse is that in an effort to work around this problem, I've seen it mentioned in chat to use poetry run pytest ... in the tox env commands, which may result in a successful test run using the poetry-managed virtualenv, but has in fact circumvented tox's purpose of testing the package from a source install in a completely isolated environment.

The cause of this bug is at https://github.com/python-poetry/poetry/blob/1.1.0b2/poetry/utils/env.py#L348:

if not in_venv or env is not None:
    ...

When this bug occurs, the situation is that there is an activated venv, so not in_venv is False, but since an out-of-project env was also found, env is not None is True so it enters this block anyway instead of going to the code that handles an activated venv. Note: an in-project venv is not checked until within this block - this special treatment prevents the bug from occurring for an in-project venv. I believe that proper functionality requires an activated venv to be honored over a Poetry-managed venv regardless of in-project or out-of-project.

I may be able to find some time to help with this, but I wanted to first bring this up and get buy-in that that would be the right behavior, or invite other options.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in poetry/utils/env.py around line 348 and reproduce the interaction between an activated external virtualenv and an out-of-project Poetry-managed environment. Verify the behavior with poetry env info -p and the tox scenario; done means the activated virtualenv remains selected while the managed environment is also present.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.