minimize storage impact from logging
- Dominant language
- Python
- Stars
- 15
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
#122 enabled logging into a file at DEBUG level, which could come of great use for troubleshooting in case of errors.
But each log file is about 19MB and in a day we already amassed 0.5GB of them. Clearly it is a bit "wasteful" and we should do something about that. Some ideas
- keep the log only if run errored out and otherwise remove (should probably be possible to setup I bet)
- gzipping folds it to ~10% of its size. May be we should store them gzipped anyways -- why to waste space (although IIRC this partition also has file system level compression enabled)
@jwodder -- wdyt?
Contributor guide
No contributing guide indexed for this repository
Research direction
Review issue #122 and locate the logging configuration that enabled DEBUG-level file logging. Measure the current log growth, then determine how retention or compression should reduce accumulated storage while preserving logs for failed runs; done means the agreed behavior is implemented and verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100