pytest-dev / pytest-dev/pytest
Should warn when using `pytest_plugins` in a test file (since it's also not the root conftest.py)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
Description
If I load a plugin using pytest_plugins in a test file, pytest should issue a warning similar to the warning given when using pytest_plugins in a non-root conftest.py.
I'm opening this as a bug because this seems unintentional. When pytest started discouraging pytest_plugins in non-root conftest.py, it should have been discouraged in test files too. If folks would prefer this as a feature request, I'm happy to change it.
Why this is a problem
The same issues with plugins in non-root conftest.py : The plugin gets activated globally for the rest of the test run, even though it seems like it'd only be activated for that one test file. (#3625 and #2062 have some discussion on pitfalls of pytest_plugins outside root conftest.py)
Moreover, since there's now a warning/error when using pytest_plugins in non-root conftest.py, the lack of a warning when using pytest_plugins in test files implies that using pytest_plugins in test files doesn't have the same problems.
Of course, the problems still apply to test files. From my experiments (using pytest 8.3.4):
- plugin fixtures are available in all test files (even if requested before running the test file with
pytest_plugins) - autouse fixtures are used in all tests that execute after the test file with
pytest_pluginsexecutes - hooks get registered/run in all test files (including the ones run before the file with
pytest_plugins)
And then if the file with pytest_plugins is omitted from the test run (e.g., by selecting a specific file), then the plugin is not loaded at all. (This behavior is reasonable/good, but it can cause surprising/confusing test failures when combined with plugins in test files.)
Minimal example
Expand for code snippets
# content of tests/some_plugin.py
import pytest
@pytest.fixture()
def some_fixture():
return "some value"
# content of tests/test_1_before_plugin.py
def test_some_fixture(some_fixture):
assert some_fixture == "some value"
# content of tests/test_2_load_plugin.py
pytest_plugins = ["tests.some_plugin"]
def test_some_fixture(some_fixture):
assert some_fixture == "some value"
# content of tests/test_3_after_plugin.py
def test_some_fixture(some_fixture):
assert some_fixture == "some value"
$ pytest tests
All tests will pass, and no warning is emitted. (This template can be tweaked to experiment with the other behaviors I described.)
Other info
- pytest: 8.3.4
- OS: NixOS
$ pip list
Package Version Editable project location
---------------------- ------- ----------------------------------------------------------
attrs 24.2.0
fancycompleter 0.9.1
iniconfig 2.0.0
packaging 24.2
pdbpp 0.10.3
pip 23.2.1
pluggy 1.5.0
Pygments 2.18.0
pyrepl 0.9.0
pytest 8.3.4
pytest-fixture-scoping 0.1.0 [...]/bug-repros/pytest-fixture-scoping
setuptools 68.2.0
wmctrl 0.5
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the behavior with tests/test_1_before_plugin.py, tests/test_2_load_plugin.py, and tests/test_3_after_plugin.py, using pytest tests as the first run. Trace how pytest_plugins is handled in a test file and compare it with the existing warning for non-root conftest.py. Done means a warning is emitted for the test-file case and coverage verifies the documented plugin-loading behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100