pytest-dev / pytest-dev/pytest

Data stored in test instance objects is never released

Open
#11,374 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: performance
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Description

I have a test suite that started life in nosetests, and a lot of the tests still use xunit style with a setup_method that creates fixture objects and stores them in the instance of the test class. I ran into a problem with tests exhausting resources, and it seems like pytest is hanging on to references to the instances of the test classes, preventing these fixture objects from being garbage collected.

Here's a proof-of-concept:

conftest.py:

import gc

def pytest_runtest_logfinish(nodeid, location):
    import test_leak
    gc.collect()
    print(f"\nExpensive objects: {test_leak.ExpensiveObject.count}", end="")

test_leak.py:

import gc
from typing import ClassVar

import pytest


class ExpensiveObject:
    count: ClassVar[int] = 0

    def __init__(self) -> None:
        ExpensiveObject.count += 1

    def __del__(self) -> None:
        ExpensiveObject.count -= 1


class TestClass:
    def setup_method(self) -> None:
        self.obj = ExpensiveObject()

    @pytest.mark.parametrize("x", range(5))
    def test(self, x: int) -> None:
        pass

Output:

test_leak.py::TestClass::test[0] PASSED                                                                                                   [ 20%]
Expensive objects: 1
test_leak.py::TestClass::test[1] PASSED                                                                                                   [ 40%]
Expensive objects: 2
test_leak.py::TestClass::test[2] PASSED                                                                                                   [ 60%]
Expensive objects: 3
test_leak.py::TestClass::test[3] PASSED                                                                                                   [ 80%]
Expensive objects: 4
test_leak.py::TestClass::test[4] PASSED                                                                                                   [100%]
Expensive objects: 5

So despite the calls to gc.collect(), the number of ExpensiveObjects is increasing. In contrast, if I use a pytest.fixture, they are all cleaned up immediately.

Environment

Package        Version
-------------- -------
exceptiongroup 1.1.3
iniconfig      2.0.0
packaging      23.1
pip            23.2.1
pluggy         1.3.0
pytest         7.4.0
setuptools     68.1.2
tomli          2.0.1
wheel          0.41.2
  • Pytest version: 7.4.0
  • OS version: Ubuntu 22.04
  • Python version: 3.10.12
  • a detailed description of the bug or problem you are having
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the behavior using conftest.py and test_leak.py, starting with pytest_runtest_logfinish and the parametrized setup_method case. Trace how pytest retains test class instances across test calls, then run the example with gc.collect() to verify that ExpensiveObjects are released after each test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.