pytest-dev / pytest-dev/pytest

Test failure when /tmp/setup.py exists

Open
#13,822 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: collection type: enhancement
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

I was debugging multiple test failures which I could reduce to a failure showing already in testing/test_warnings.py

pytest -s testing/test_warnings.py -k test_warning_recorded_hook
============ FAILURES ==================
_______________ test_warning_recorded_hook ____________________

pytester = <Pytester PosixPath('/tmp/pytest-of-s3248973/pytest-12/test_warning_recorded_hook0')>

>   ???
E   AssertionError: [('config warning', 'config', '', ('/tmp/pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/conftest.py', 2, 'pytest_configure')), ('collect warning', 'collect', '', None), ('setup warning', 'runtest', 'pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func', None), ('call warning', 'runtest', 'pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func', None), ('teardown warning', 'runtest', 'pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func', None)]
E   assert 'pytest-of-s3...py::test_func' == 'test_warning...py::test_func'
E     - test_warning_recorded_hook.py::test_func
E     + pytest-of-s3248973/pytest-12/test_warning_recorded_hook0/test_warning_recorded_hook.py::test_func

I.e. for the recording of the warning location it uses the path under /tmp instead of only the filename.

Tracing further the outmost Collector has a nodeid of pytest-of-s3248973/pytest-12/test_warning_recorded_hook0 instead of test_warning_recorded_hook.py because session.config.rootpath is determined as /tmp, see https://github.com/pytest-dev/pytest/blob/c97a40140b2eca3ea45fcb1645d77da8cae3ee32/src/_pytest/nodes.py#L601

Following _pytest.config._initini to determine_setup to this loop https://github.com/pytest-dev/pytest/blob/c97a40140b2eca3ea45fcb1645d77da8cae3ee32/src/_pytest/config/findpaths.py#L222-L225

It starts from the test folder ancestor where the test is invoked from upwards until it finds /tmp/setup.py and then treats /tmp as the rootdir

This ultimately causes the failure.

I see 2 solutions: Ignore tempfile.gettempdir()/setup.py and/or make the tests resilient by either passing --rootdir and/or matching against path suffixes instead of full paths in the test assertions

I'm using pytest 7.2.2 w/ Python 3.9 but it should affect (almost) any version

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with pytest -s testing/test_warnings.py -k test_warning_recorded_hook while /tmp/setup.py exists. Read src/_pytest/nodes.py, _pytest.config._initini, determine_setup, and the findpaths.py loop described in the issue; done means the warning location assertion no longer fails in this setup, with regression coverage or resilient assertions as appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.