borgbackup / borgbackup/borgstore
caching: add continuous cache eviction
- Dominant language
- Python
- Stars
- 25
- Forks
- 8
- Avg merge
- 16m
- Merged PRs (30d)
- 8
Description
Currently, cache eviction is only done at `Store` open/close time (IIRC).
Problem:
If the cache is enabled (e.g. `C_WRITETHROUGH` and a limited cache size [== a full mirror is not desired]), reading a lot of data from the store will use up as much disk space as the (deduplicated) size of the loaded objects.
When using partial loads it is even worse, as it always caches the full object, not just the range requested.
So, guess we need to do cache eviction more frequently, but not on every request.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the Store open/close eviction paths and the read and partial-load paths that populate the cache. Determine a periodic eviction policy that avoids running on every request while keeping the configured cache size bounded; done means repeated reads do not grow cached disk usage beyond that limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100