pytest-dev / pytest-dev/pytest

Should warn when using `pytest_plugins` in a test file (since it's also not the root conftest.py)

Open
#13,030 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: deprecation
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

  • a detailed description of the bug or problem you are having
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible
Description

If I load a plugin using pytest_plugins in a test file, pytest should issue a warning similar to the warning given when using pytest_plugins in a non-root conftest.py.

I'm opening this as a bug because this seems unintentional. When pytest started discouraging pytest_plugins in non-root conftest.py, it should have been discouraged in test files too. If folks would prefer this as a feature request, I'm happy to change it.

Why this is a problem

The same issues with plugins in non-root conftest.py : The plugin gets activated globally for the rest of the test run, even though it seems like it'd only be activated for that one test file. (#3625 and #2062 have some discussion on pitfalls of pytest_plugins outside root conftest.py)

Moreover, since there's now a warning/error when using pytest_plugins in non-root conftest.py, the lack of a warning when using pytest_plugins in test files implies that using pytest_plugins in test files doesn't have the same problems.

Of course, the problems still apply to test files. From my experiments (using pytest 8.3.4):

  • plugin fixtures are available in all test files (even if requested before running the test file with pytest_plugins)
  • autouse fixtures are used in all tests that execute after the test file with pytest_plugins executes
  • hooks get registered/run in all test files (including the ones run before the file with pytest_plugins)

And then if the file with pytest_plugins is omitted from the test run (e.g., by selecting a specific file), then the plugin is not loaded at all. (This behavior is reasonable/good, but it can cause surprising/confusing test failures when combined with plugins in test files.)

Minimal example
Expand for code snippets

# content of tests/some_plugin.py

import pytest

@pytest.fixture()
def some_fixture():
    return "some value"
# content of tests/test_1_before_plugin.py

def test_some_fixture(some_fixture):
    assert some_fixture == "some value"
# content of tests/test_2_load_plugin.py

pytest_plugins = ["tests.some_plugin"]

def test_some_fixture(some_fixture):
    assert some_fixture == "some value"
# content of tests/test_3_after_plugin.py

def test_some_fixture(some_fixture):
    assert some_fixture == "some value"
$ pytest tests

All tests will pass, and no warning is emitted. (This template can be tweaked to experiment with the other behaviors I described.)

Other info
  • pytest: 8.3.4
  • OS: NixOS
$ pip list
Package                Version Editable project location
---------------------- ------- ----------------------------------------------------------
attrs                  24.2.0
fancycompleter         0.9.1
iniconfig              2.0.0
packaging              24.2
pdbpp                  0.10.3
pip                    23.2.1
pluggy                 1.5.0
Pygments               2.18.0
pyrepl                 0.9.0
pytest                 8.3.4
pytest-fixture-scoping 0.1.0   [...]/bug-repros/pytest-fixture-scoping
setuptools             68.2.0
wmctrl                 0.5

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the behavior with tests/test_1_before_plugin.py, tests/test_2_load_plugin.py, and tests/test_3_after_plugin.py, using pytest tests as the first run. Trace how pytest_plugins is handled in a test file and compare it with the existing warning for non-root conftest.py. Done means a warning is emitted for the test-file case and coverage verifies the documented plugin-loading behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.