allenai / allenai/cached_path

Add cache limits

Open
#258 0 comments 0 reactions 0 assignees View on GitHub
Feature request
Dominant language
Python
Stars
47
Forks
22
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
Currently the size of the cache directory can grow unbounded. This can cause problems (e.g. disk full) and require manual intervention (e.g. manually deleting the entire cache directory).

**Describe the solution you'd like**
Add size/age limits to the cache directory and the ability for users to enforce those limits by deleting the least recently used data with a function call. Similar to the "reduce_size" function in joblib.memory ([link](https://joblib.readthedocs.io/en/latest/generated/joblib.Memory.html#joblib.Memory.reduce_size)).

**Describe alternatives you've considered**
A cron job to automatically remove the entire cache direction is a hack and seems like overkill.

**Additional context**
None

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.