pytest-dev / pytest-dev/pytest

The documented pattern for parametrizing conditional raising leaks a lot of memory

Open
#13,410 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: performance
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

The documented pattern for parametrizing conditional raising leaks a lot of memory.

The problem is that the pytest.raises context manager is mutated with a reference to the raised exception and traceback when exiting the context manager:
https://github.com/pytest-dev/pytest/blob/923044785f9fb7b1fac2c5d52df1fb8ea84ffc37/src/_pytest/python_api.py#L1047-L1048

This causes the traceback to be kept in memory until the test session terminates.

If this is indeed the correct way for this kind of parametrization, pytest should do some automatic clean up after the test finishes.

Otherwise, the documented parametrization should be adjusted to not pass in mutable objects, for example:

@pytest.mark.parametrize(
    "example_input,expectation",
    [
        (3, lambda: nullcontext(2)),
        (2, lambda: nullcontext(3)),
        (1, lambda: nullcontext(6)),
        (0, lambda: pytest.raises(ZeroDivisionError)),
    ],
)
def test_division(example_input, expectation):
    """Test how much I know division."""
    with expectation() as e:
        assert (6 / example_input) == e

The same problem happens if the parametrization involves exception objects which are raised during the test, the objects are mutated by Python which adds a __traceback__ to them:

@pytest.mark.parametrize("error", [Exception("Boom!"),])
def test_division(error):
    """Blow up."""
    raise error

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/_pytest/python_api.py at the pytest.raises context-manager code linked in the issue, then compare it with the documented conditional-raising parametrization. Reproduce the retained traceback with the examples in the report and determine whether cleanup or a documentation change is appropriate; done means the parametrized test no longer retains the traceback unexpectedly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.