pytest-dev / pytest-dev/pytest
Cannot use python logging without maintaining all records in caplog
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Description
I'd like to be able to use python's built-in logging, specifically to stream logging records to CLI and file, without having all log records stored in memory in the caplog fixture.
The reason I don't want the records to be stored in caplog is that this is essentially a forced memory leak. In the use-case where I discovered this, all DEBUG-level logging records were being stored persistently in caplog even though my CLI logging level was INFO, file handler level was DEBUG. The records were being streamed to file as expected, but I don't want them kept in memory. This was a very long-running test, the accumulating DEBUG records eventually resulted in a memory error.
Maybe I just don't know how to use caplog/logging in such a way that I can handle my logging streams as desired without keeping the log records in memory. If that's the case, would it be possible to get that clarification?
Environment
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 23 2018, 23:31:17) [MSC v.1916 32 bit (Intel)]
system='Windows', release='10', version='10.0.16299', machine='AMD64', processor='Intel64 Family 6 Model 142 Stepping 10, GenuineIntel'
Package Version
------------------ -------
atomicwrites 1.4.0
attrs 20.3.0
colorama 0.4.4
importlib-metadata 3.4.0
iniconfig 1.1.1
packaging 20.9
pip 20.3.3
pluggy 0.13.1
py 1.10.0
pyparsing 2.4.7
pytest 6.2.2
setuptools 51.0.0
toml 0.10.2
typing-extensions 3.7.4.3
wheel 0.36.2
zipp 3.4.0
Example
import logging
logger = logging.getLogger(__name__)
test_logging_ex(caplog):
assert not caplog.records
logger.info("I'd like to see this message in the CLI, but I can't"
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the provided caplog example with Python logging and inspect how the caplog fixture retains records while CLI and file handlers process them. Done means logging can be streamed to the requested destinations without retaining all records in memory, with behavior covered for the long-running test case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100