drivendataorg / drivendataorg/cloudpathlib

Add a limit to the size of the local cache

Open
#11 0 comments 0 reactions 0 assignees View on GitHub
caching enhancement
Dominant language
Python
Stars
628
Forks
88
Avg merge
17h 28m
Merged PRs (30d)
2

Description

Is there a package that will implement this for us?

We may need to create an LRU cache that we check the size of and remove files before downloading more. The tricky part about this being LRU cache is that we manipulate `atime` and `mtime` to match the cloud exactly. Perhaps we can abuse ctime to store the cached date so that we can sort by what has been downloaded most recently? Or maybe we stop overriding `atime` and trust that?

Also needs research: are there fast platform independent ways to estimate size of a directory?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.