pytest-dev / pytest-dev/pytest
Memory leak when using numpy matrix in tests
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Hi, when using non-empty numpy matrices in tests they will not be garbage collected and pile up slowly using more and more memory.
pip list output:
attrs 22.2.0
exceptiongroup 1.1.1
iniconfig 2.0.0
numpy 1.24.2
packaging 23.0
pip 10.0.1
pluggy 1.0.0
pytest 7.2.2
setuptools 39.1.0
tomli 2.0.1
Operating system:
Ubuntu 18.04.6 LTS
python 3.8
Minimal example:
import numpy as np
class TestMemory:
def test_memory(self):
for _ in range(1000000):
test = np.matrix(np.random.rand(100, 100))
np.array does not share the same issue, so one can avoid matrix as much as possible, but for some operations it is still needed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the minimal TestMemory example with the listed Python, pytest, and numpy versions, then compare its memory behavior with the np.array variant. Investigate why non-empty numpy.matrix values are retained during tests. Done means repeated matrix creation no longer causes memory usage to accumulate, while the relevant test behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- performance, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100