fsspec / fsspec/filesystem_spec
cache and forget mode?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
There can be a case such as: a single large remote zip file, from which we would like to extract and cache files for further access. In this case, it might well be better to simply download the remote file and do the extraction locally. You could do this now with:
"simplecache::zip://myfiles*::simplecache::s3://bucket/file.zip"
Note that "simplecache" appears twice, but we don't actually want to store the compressed file. Should there be an even simpler "tempcache" that throws away files once nothing depends on them any more?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue discusses chaining simplecache with zip and s3 to extract files from a remote archive without retaining the compressed file. No source file, test, or entry point is named, so begin by locating the simplecache implementation and its lifecycle tests. Done would require a decided tempcache design and tests showing when cached files are discarded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100