pytest-dev / pytest-dev/pytest

Scopes of several files are merged when specifying a config using `-c`

Open
#9,703 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: critical topic: collection type: bug type: regression
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Setup:

Python 3.9.9, pip list:

attrs      21.4.0
iniconfig  1.1.1
packaging  21.3
pip        21.3.1
pluggy     1.0.0
py         1.11.0
pyparsing  3.0.7
pytest     7.0.1
setuptools 60.5.0
tomli      2.0.1
wheel      0.37.1

(basically just whatever pip install pytest installs)

Files:

# <dir>/test/test_file1.py

import pytest

@pytest.fixture(autouse=True)
def some_fixture():
    print("Fixture called")

def test_in_file1():
    print("test_in_file1")
# <dir>/test/test_file2.py

def test_in_file2():
    print("test_in_file2")
# <dir>/config/pytest.ini
[pytest]

Running with specifying the config from config explicitly executes the fixture for both tests:
py.test -c config/pytest.ini -s -v tests/test_file1.py tests/test_file2.py

config/::test_in_file1 Fixture called
test_in_file1
PASSED
config/::test_in_file2 Fixture called
test_in_file2
PASSED

Omitting the config, using a config from the root directory <dir>, or not specifying the test files explicitly (just py.test -c config/pytest.ini -s -v):

config/tests/test_file1.py::test_in_file1 Fixture called
test_in_file1
PASSED
config/tests/test_file2.py::test_in_file2 test_in_file2
PASSED

which is the behavior I would expect.

Downgrading to pytest==6.2.5 and running the command above (py.test -c config/pytest.ini -s -v tests/test_file1.py tests/test_file2.py) does not lead to the bug either. Interestingly enough, the "config" prefix is missing from the output:

tests/test_file1.py::test_in_file1 Fixture called
test_in_file1
PASSED
tests/test_file2.py::test_in_file2 test_in_file2
PASSED

Must be some config-based grouping feature introduced in 7?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with test/test_file1.py, test/test_file2.py, and config/pytest.ini using pytest 7.0.1, then compare the explicit -c invocation with pytest 6.2.5 and the other commands described. Trace how the selected config affects fixture scope and test paths; done means the fixture runs only for test_file1.py while the explicit-config command still reports the expected tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.