pytest-dev / pytest-dev/pytest

tmp_path_retention_policy="failed" removes directories for setup and teardown errors

Open
#14,998 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

  • a detailed description of the bug or problem you are having
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible
Description

With tmp_path_retention_policy = "failed", temporary directories are retained for failures during the test call, but removed when an error occurs during fixture setup or teardown.

The documentation describes "failed" as retaining directories for tests whose outcome is "error" or "failed", so I expected all three cases to be retained.

I reproduced this locally on main at commit 3fd8675d6d798507c06cf9c60753be6d9d7b0e17. I have not verified released versions.

Reproduction

pytest.ini:

[pytest]
tmp_path_retention_policy = failed

test_repro.py:

import pytest


def write_evidence(tmp_path, phase):
    evidence = tmp_path / "debug-evidence.txt"
    evidence.write_text(f"keep me: {phase}", encoding="utf-8")
    print(f"created[{phase}]={evidence} exists={evidence.exists()}")


@pytest.fixture
def broken_during_setup(tmp_path):
    write_evidence(tmp_path, "setup")
    raise RuntimeError("setup failed")


def test_setup_error(broken_during_setup):
    pass


def test_call_failure(tmp_path):
    write_evidence(tmp_path, "call")
    assert False


@pytest.fixture
def broken_during_teardown(tmp_path):
    write_evidence(tmp_path, "teardown")
    yield
    raise RuntimeError("teardown failed")


def test_teardown_error(broken_during_teardown):
    pass

Run:

python -m pytest -q -s --basetemp=retained test_repro.py

All three files exist immediately after creation:

created[setup]=...\test_setup_error0\debug-evidence.txt exists=True
created[call]=...\test_call_failure0\debug-evidence.txt exists=True
created[teardown]=...\test_teardown_error0\debug-evidence.txt exists=True

1 failed, 1 passed, 2 errors

After pytest exits, list the retained evidence:

python -c "from pathlib import Path; print(*Path('retained').rglob('debug-evidence.txt'), sep='\n')"

Actual result:

retained\test_call_failure0\debug-evidence.txt

The directories for the setup and teardown errors have been removed.

Expected result

All three directories should remain:

test_setup_error0
test_call_failure0
test_teardown_error0
Possible cause

The tmp_path finalizer currently bases cleanup primarily on:

result_dict.get("call", True)

For a setup error, the call phase does not exist, so the default value causes the directory to be removed.

For a teardown error, the tmp_path finalizer runs before the teardown report has been produced. If the call phase passed, it removes the directory without taking the teardown outcome into account.

Possible direction

Would it be appropriate to register the temporary path on the item and defer the cleanup decision until the teardown report is available?

Any fix should preserve the behavior covered by #10502: a test skipped during fixture setup, without an accompanying error, should not retain its tmp_path directory under the "failed" policy.

I would be happy to work on a fix after confirming the preferred lifecycle hook.

Environment
pytest 9.2.0.dev293+g3fd8675d6
commit 3fd8675d6d798507c06cf9c60753be6d9d7b0e17
Python 3.12.10
Microsoft Windows 11 Home 10.0.26200, 64-bit
pip list
attrs==26.1.0
certifi==2026.7.22
charset-normalizer==3.5.1
colorama==0.4.6
elementpath==5.1.4
execnet==2.1.2
hypothesis==6.168.0
idna==3.19
iniconfig==2.3.0
mock==5.2.0
numpy==2.5.3
packaging==26.3
pip==26.2.1
pluggy==1.6.0
Pygments==2.21.0
pytest==9.2.0.dev293+g3fd8675d6
pytest-xdist==3.8.0
PyYAML==6.0.3
requests==2.34.2
sortedcontainers==2.4.0
urllib3==2.7.0
xmlschema==4.3.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/_pytest/tmpdir.py at the tmp_path finalizer referenced in the report, then run the provided test_repro.py with pytest.ini and the shown pytest command. Trace setup, call, and teardown reporting, including the skipped-setup behavior from #10502. Done means failed and errored setup, call, and teardown cases retain their directories while setup skips do not.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.