pytest-dev / pytest-dev/pytest

Docs on warnings filters should note that `ResourceWarning` is often delayed

Open Beginner friendly
#9,825 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: docs
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Some of my tests need to ensure open files are closed. Let's use the simplest example:

# test_example.py
import pytest

@pytest.mark.filterwarnings('error::ResourceWarning')
def test_resourcewarning():
    open('/dev/null')

When I run pytest, the warning is thrown and even printed in the test output, but the test still passes:

$ pytest
======================================== test session starts =========================================
platform darwin -- Python 3.10.2, pytest-7.1.1, pluggy-1.0.0
rootdir: /path/to/project
collected 1 item

test_example.py .                                                                              [100%]

========================================== warnings summary ==========================================
test_example.py::test_resourcewarning
  /path/to/python/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: 
Exception ignored in: <_io.FileIO [closed]>

  Traceback (most recent call last):
    File "/path/to/project/test_example.py", line 5, in test_resourcewarning
      open('/dev/null')
  ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>

    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================== 1 passed, 1 warning in 0.01s ====================================

Removing @pytest.mark.filterwarnings or changing the warning type to something unrelated (like DeprecationWarning) results in the test passing without printing any warnings at all. That tells me pytest is picking up the warning, but it's being subsequently caught by pytest.PytestUnraisableExceptionWarning, and my tests still pass because I wasn't filtering for that. If I filter for pytest.PytestUnraisableExceptionWarning instead the test also passes, because it isn't looking for the original ResourceWarning.

The only solution I can think of is to filter for both:

@pytest.mark.filterwarnings('error::ResourceWarning')
@pytest.mark.filterwarnings('error::pytest.PytestUnraisableExceptionWarning')

Unless I'm missing something this seems to be a bug for ResourceWarning in particular, since I can't reproduce this with other warning types. I think failing the test case without requiring the 2nd generic warning filter is the reasonable expected behaviour here.

Note: this test example was run on a vanilla virtual env:

$ pip freeze
attrs==21.4.0
iniconfig==1.1.1
packaging==21.3
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.7
pytest==7.1.1
tomli==2.0.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the test_example.py reproduction and the pytest warnings documentation referenced in the report; run the example to confirm the delayed ResourceWarning behavior. Update the warnings-filter documentation to explain this behavior and the relevant filtering expectation, then verify the documented example and surrounding tests still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, testing
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.