fsspec / fsspec/filesystem_spec
Adaptive Prefetcher on top of different cache-types
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
Hi @martindurant,
Creating this issues to take an alignment before making any changes to move prefetcher logic (from gcsfs) to filesystem_spec repo:
- Do you want to keep prefetcher as a new cache-type or want to keep over the existing caches?
- If on top of the caches, do you have any idea about the end state or how you want to go about that?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading gcsfs/prefetcher.py, the implementation named in the issue, and compare it with the existing cache types in filesystem_spec. The first step is to resolve whether the prefetcher should be a new cache type or operate above existing caches; completion criteria are not defined until that design direction is agreed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100