pytest-dev / pytest-dev/pytest
Test failure when /tmp/setup.py exists
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
I was debugging multiple test failures which I could reduce to a failure showing already in testing/test_warnings.py
pytest -s testing/test_warnings.py -k test_warning_recorded_hook
============ FAILURES ==================
_______________ test_warning_recorded_hook ____________________
pytester = <Pytester PosixPath('/tmp/pytest-of-s3248973/pytest-12/test_warning_recorded_hook0')>
> ???
E AssertionError: [('config warning', 'config', '', ('/tmp/pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/conftest.py', 2, 'pytest_configure')), ('collect warning', 'collect', '', None), ('setup warning', 'runtest', 'pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func', None), ('call warning', 'runtest', 'pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func', None), ('teardown warning', 'runtest', 'pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func', None)]
E assert 'pytest-of-s3...py::test_func' == 'test_warning...py::test_func'
E - test_warning_recorded_hook.py::test_func
E + pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func
I.e. for the recording of the warning location it uses the path under /tmp instead of only the filename.
Tracing further the outmost Collector has a nodeid of pytest-of-s3248973/pytest-12/test_warning_recorded_hook0 instead of test_warning_recorded_hook.py because session.config.rootpath is determined as /tmp, see https://github.com/pytest-dev/pytest/blob/c97a40140b2eca3ea45fcb1645d77da8cae3ee32/src/_pytest/nodes.py#L601
Following _pytest.config._initini to determine_setup to this loop https://github.com/pytest-dev/pytest/blob/c97a40140b2eca3ea45fcb1645d77da8cae3ee32/src/_pytest/config/findpaths.py#L222-L225
It starts from the test folder ancestor where the test is invoked from upwards until it finds /tmp/setup.py and then treats /tmp as the rootdir
This ultimately causes the failure.
I see 2 solutions: Ignore tempfile.gettempdir()/setup.py and/or make the tests resilient by either passing --rootdir and/or matching against path suffixes instead of full paths in the test assertions
I'm using pytest 7.2.2 w/ Python 3.9 but it should affect (almost) any version
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with pytest -s testing/test_warnings.py -k test_warning_recorded_hook while /tmp/setup.py exists. Read src/_pytest/nodes.py, _pytest.config._initini, determine_setup, and the findpaths.py loop described in the issue; done means the warning location assertion no longer fails in this setup, with regression coverage or resilient assertions as appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100