man-group / man-group/pytest-plugins

No module named virtualenv when running under PYTHONPATH

Open
#165 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
597
Forks
89
PR merge metrics
No merged PRs in 30d

Description

Easiest way to reproduce is with pip-run:

```
$ cat > test.py
__requires__ = ['pytest-virtualenv']

def test_run(virtualenv):
pass
$ pip-run -- -m pytest test.py
Collecting pytest-virtualenv
Using cached pytest_virtualenv-1.7.0-py2.py3-none-any.whl (9.5 kB)
Collecting pytest-fixture-config
Using cached pytest_fixture_config-1.7.0-py2.py3-none-any.whl (6.5 kB)
Collecting virtualenv
Using cached virtualenv-20.0.30-py2.py3-none-any.whl (7.1 MB)
Collecting pytest
Using cached pytest-6.0.1-py3-none-any.whl (270 kB)
Collecting pytest-shutil
Using cached pytest_shutil-1.7.0-py2.py3-none-any.whl (15 kB)
Collecting appdirs<2,>=1.4.3
Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting filelock<4,>=3.0.0
Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting six<2,>=1.9.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting distlib<1,>=0.3.1
Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting iniconfig
Using cached iniconfig-1.0.1-py3-none-any.whl (4.2 kB)
Collecting toml
Using cached toml-0.10.1-py2.py3-none-any.whl (19 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting packaging
Using cached packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting py>=1.8.2
Using cached py-1.9.0-py2.py3-none-any.whl (99 kB)
Collecting attrs>=17.4.0
Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.4.0-py3-none-any.whl (43 kB)
Collecting execnet
Using cached execnet-1.7.1-py2.py3-none-any.whl (39 kB)
Collecting path.py
Using cached path.py-12.5.0-py3-none-any.whl (2.3 kB)
Processing /Users/jaraco/Library/Caches/pip/wheels/7c/06/54/bc84598ba1daf8f970247f550b175aaaee85f68b4b0c5ab2c6/termcolor-1.1.0-cp38-none-any.whl
Collecting mock
Using cached mock-4.0.2-py3-none-any.whl (28 kB)
Collecting contextlib2
Using cached contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)
Collecting pyparsing>=2.0.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting apipkg>=1.4
Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)
Collecting path
Using cached path-15.0.0-py3-none-any.whl (21 kB)
Installing collected packages: iniconfig, toml, pluggy, pyparsing, six, packaging, py, attrs, more-itertools, pytest, pytest-fixture-config, appdirs, filelock, distlib, virtualenv, apipkg, execnet, path, path.py, termcolor, mock, contextlib2, pytest-shutil, pytest-virtualenv
Successfully installed apipkg-1.5 appdirs-1.4.4 attrs-19.3.0 contextlib2-0.6.0.post1 distlib-0.3.1 execnet-1.7.1 filelock-3.0.12 iniconfig-1.0.1 mock-4.0.2 more-itertools-8.4.0 packaging-20.4 path-15.0.0 path.py-12.5.0 pluggy-0.13.1 py-1.9.0 pyparsing-2.4.7 pytest-6.0.1 pytest-fixture-config-1.7.0 pytest-shutil-1.7.0 pytest-virtualenv-1.7.0 six-1.15.0 termcolor-1.1.0 toml-0.10.1 virtualenv-20.0.30
WARNING: You are using pip version 20.2; however, version 20.2.1 is available.
You should consider upgrading via the '/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -m pip install --upgrade pip' command.
=========================================================== test session starts ============================================================
platform darwin -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/jaraco/draft
plugins: shutil-1.7.0, virtualenv-1.7.0, black-multipy-1.0.0, black-0.3.8, flake8-1.0.5, checkdocs-1.2.3, cov-2.8.1
collected 1 item

test.py E [100%]

================================================================== ERRORS ==================================================================
________________________________________________________ ERROR at setup of test_run ________________________________________________________

args = (), kwargs = {}, var = 'virtualenv_executable', gen =

@functools.wraps(f)
def wrapper(*args, **kwargs):
for var in vars_:
if not getattr(cfg, var):
pytest.skip('config variable {0} missing, skipping test'.format(var))
gen = f(*args, **kwargs)
> yield next(gen)

/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-hirmpjlp/pytest_fixture_config.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-hirmpjlp/pytest_virtualenv.py:50: in virtualenv
venv = VirtualEnv()
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-hirmpjlp/pytest_virtualenv.py:143: in __init__
self.run(cmd)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-hirmpjlp/pytest_virtualenv.py:151: in run
return super(VirtualEnv, self).run(args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self =
cmd = ['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '-m', 'virtualenv', '-p', '/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmp7n3bqfab/.env']
capture = False, check_rc = True, cd = Path('/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmp7n3bqfab'), shell = False
kwargs = {'env': {'BASH_COMPLETIONS': '/usr/local/share/bash-completion/bash_completion:/usr/local/etc/bash_completion', 'BASH_COMPLETION_COMPAT_DIR': '/usr/local/etc/bash_completion.d', 'COLORTERM': 'truecolor', 'EDITOR': 'subl --wait', ...}}
p = , out = None, _ = None
err = CalledProcessError(1, ['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '-m', 'virtualenv', '-p', '/...rks/Python.framework/Versions/3.8/bin/python3.8', '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmp7n3bqfab/.env'])

def run(self, cmd, capture=False, check_rc=True, cd=None, shell=False, **kwargs):
"""
Run a command relative to a given directory, defaulting to the workspace root

Parameters
----------
cmd : `str` or `list`
Command string or list. Commands given as a string will be run in a subshell.
capture : `bool`
Capture and return output
check_rc : `bool`
Assert return code is zero
cd : `str`
Path to chdir to, defaults to workspace root
"""
if isinstance(cmd, string_types):
shell = True
else:
# Some of the command components might be path objects or numbers
cmd = [str(i) for i in cmd]

if not cd:
cd = self.workspace

with cmdline.chdir(cd):
log.debug("run: {0}".format(cmd))
if capture:
p = subprocess.Popen(cmd, shell=shell, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **kwargs)
else:
p = subprocess.Popen(cmd, shell=shell, **kwargs)
(out, _) = p.communicate()

if out is not None and not isinstance(out, string_types):
out = out.decode('utf-8')

if self.debug and capture:
log.debug("Stdout/stderr:")
log.debug(out)

if check_rc and p.returncode != 0:
err = subprocess.CalledProcessError(p.returncode, cmd)
err.output = out
if capture and not self.debug:
log.error("Stdout/stderr:")
log.error(out)
> raise err
E subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '-m', 'virtualenv', '-p', '/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmp7n3bqfab/.env']' returned non-zero exit status 1.

/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-hirmpjlp/pytest_shutil/workspace.py:132: CalledProcessError
---------------------------------------------------------- Captured stderr setup -----------------------------------------------------------
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: No module named virtualenv
========================================================= short test summary info ==========================================================
ERROR test.py::test_run - subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '-m...
============================================================= 1 error in 0.11s =============================================================
```

To reproduce without pip-run:

```
$ cat > test.py
def test_run(virtualenv):
pass
$ pip install -t libs pytest-virtualenv
Collecting pytest-virtualenv
Using cached pytest_virtualenv-1.7.0-py2.py3-none-any.whl (9.5 kB)
Collecting pytest-fixture-config
Using cached pytest_fixture_config-1.7.0-py2.py3-none-any.whl (6.5 kB)
Collecting virtualenv
Using cached virtualenv-20.0.30-py2.py3-none-any.whl (7.1 MB)
Collecting pytest
Using cached pytest-6.0.1-py3-none-any.whl (270 kB)
Collecting pytest-shutil
Using cached pytest_shutil-1.7.0-py2.py3-none-any.whl (15 kB)
Collecting distlib<1,>=0.3.1
Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting six<2,>=1.9.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting appdirs<2,>=1.4.3
Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting filelock<4,>=3.0.0
Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting importlib-metadata<2,>=0.12; python_version < "3.8"
Using cached importlib_metadata-1.7.0-py2.py3-none-any.whl (31 kB)
Collecting iniconfig
Using cached iniconfig-1.0.1-py3-none-any.whl (4.2 kB)
Collecting attrs>=17.4.0
Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.4.0-py3-none-any.whl (43 kB)
Collecting py>=1.8.2
Using cached py-1.9.0-py2.py3-none-any.whl (99 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting toml
Using cached toml-0.10.1-py2.py3-none-any.whl (19 kB)
Collecting packaging
Using cached packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting mock
Using cached mock-4.0.2-py3-none-any.whl (28 kB)
Collecting path.py
Using cached path.py-12.5.0-py3-none-any.whl (2.3 kB)
Processing /Users/jaraco/Library/Caches/pip/wheels/7c/06/54/bc84598ba1daf8f970247f550b175aaaee85f68b4b0c5ab2c6/termcolor-1.1.0-cp37-none-any.whl
Collecting contextlib2
Using cached contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)
Collecting execnet
Using cached execnet-1.7.1-py2.py3-none-any.whl (39 kB)
Collecting zipp>=0.5
Using cached zipp-3.1.0-py3-none-any.whl (4.9 kB)
Collecting pyparsing>=2.0.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting path
Using cached path-15.0.0-py3-none-any.whl (21 kB)
Collecting apipkg>=1.4
Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)
Installing collected packages: iniconfig, attrs, more-itertools, py, zipp, importlib-metadata, pluggy, toml, pyparsing, six, packaging, pytest, pytest-fixture-config, distlib, appdirs, filelock, virtualenv, mock, path, path.py, termcolor, contextlib2, apipkg, execnet, pytest-shutil, pytest-virtualenv
Successfully installed apipkg-1.5 appdirs-1.4.4 attrs-19.3.0 contextlib2-0.6.0.post1 distlib-0.3.1 execnet-1.7.1 filelock-3.0.12 importlib-metadata-1.7.0 iniconfig-1.0.1 mock-4.0.2 more-itertools-8.4.0 packaging-20.4 path-15.0.0 path.py-12.5.0 pluggy-0.13.1 py-1.9.0 pyparsing-2.4.7 pytest-6.0.1 pytest-fixture-config-1.7.0 pytest-shutil-1.7.0 pytest-virtualenv-1.7.0 six-1.15.0 termcolor-1.1.0 toml-0.10.1 virtualenv-20.0.30 zipp-3.1.0
WARNING: You are using pip version 20.2; however, version 20.2.1 is available.
You should consider upgrading via the '/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 -m pip install --upgrade pip' command.
$ env PYTHONPATH=libs python -m pytest test.py
=========================================================== test session starts ============================================================
platform darwin -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/jaraco/draft
plugins: shutil-1.7.0, virtualenv-1.7.0, black-multipy-1.0.0, black-0.3.8, flake8-1.0.5, checkdocs-1.2.3, cov-2.8.1
collected 1 item

test.py E [100%]

================================================================== ERRORS ==================================================================
________________________________________________________ ERROR at setup of test_run ________________________________________________________

args = (), kwargs = {}, var = 'virtualenv_executable', gen =

@functools.wraps(f)
def wrapper(*args, **kwargs):
for var in vars_:
if not getattr(cfg, var):
pytest.skip('config variable {0} missing, skipping test'.format(var))
gen = f(*args, **kwargs)
> yield next(gen)

libs/pytest_fixture_config.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
libs/pytest_virtualenv.py:50: in virtualenv
venv = VirtualEnv()
libs/pytest_virtualenv.py:143: in __init__
self.run(cmd)
libs/pytest_virtualenv.py:151: in run
return super(VirtualEnv, self).run(args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self =
cmd = ['/usr/local/bin/python', '-m', 'virtualenv', '-p', '/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmporcm9r1f/.env']
capture = False, check_rc = True, cd = Path('/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmporcm9r1f'), shell = False
kwargs = {'env': {'BASH_COMPLETIONS': '/usr/local/share/bash-completion/bash_completion:/usr/local/etc/bash_completion', 'BASH_COMPLETION_COMPAT_DIR': '/usr/local/etc/bash_completion.d', 'COLORTERM': 'truecolor', 'EDITOR': 'subl --wait', ...}}
p = , out = None, _ = None
err = CalledProcessError(1, ['/usr/local/bin/python', '-m', 'virtualenv', '-p', '/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmporcm9r1f/.env'])

def run(self, cmd, capture=False, check_rc=True, cd=None, shell=False, **kwargs):
"""
Run a command relative to a given directory, defaulting to the workspace root

Parameters
----------
cmd : `str` or `list`
Command string or list. Commands given as a string will be run in a subshell.
capture : `bool`
Capture and return output
check_rc : `bool`
Assert return code is zero
cd : `str`
Path to chdir to, defaults to workspace root
"""
if isinstance(cmd, string_types):
shell = True
else:
# Some of the command components might be path objects or numbers
cmd = [str(i) for i in cmd]

if not cd:
cd = self.workspace

with cmdline.chdir(cd):
log.debug("run: {0}".format(cmd))
if capture:
p = subprocess.Popen(cmd, shell=shell, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **kwargs)
else:
p = subprocess.Popen(cmd, shell=shell, **kwargs)
(out, _) = p.communicate()

if out is not None and not isinstance(out, string_types):
out = out.decode('utf-8')

if self.debug and capture:
log.debug("Stdout/stderr:")
log.debug(out)

if check_rc and p.returncode != 0:
err = subprocess.CalledProcessError(p.returncode, cmd)
err.output = out
if capture and not self.debug:
log.error("Stdout/stderr:")
log.error(out)
> raise err
E subprocess.CalledProcessError: Command '['/usr/local/bin/python', '-m', 'virtualenv', '-p', '/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmporcm9r1f/.env']' returned non-zero exit status 1.

libs/pytest_shutil/workspace.py:132: CalledProcessError
---------------------------------------------------------- Captured stderr setup -----------------------------------------------------------
/usr/local/bin/python: No module named virtualenv
========================================================= short test summary info ==========================================================
ERROR test.py::test_run - subprocess.CalledProcessError: Command '['/usr/local/bin/python', '-m', 'virtualenv', '-p', '/Library/Framework...
============================================================= 1 error in 0.11s =============================================================
```

I suspect the issue is that pytest-virtualenv is suppressing the PYTHONPATH when executing virtualenv.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the virtualenv fixture entry point shown in pytest_virtualenv.py and the subprocess runner in pytest_shutil/workspace.py. Reproduce the failure with pip-run or a PYTHONPATH installation, then verify that the fixture can create an environment without reporting No module named virtualenv.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.