fsspec / fsspec/filesystem_spec
AbstractFileSystem is not abstract
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
I'm trying to learn more about how fsspec works (so I can better understand the IO part of the Pangeo stack), but I'm confused by the structure of fsspec/filesystem_spec.
Despite the name, it seems AbstractFileSystem is not an abstract base class (nor is AbstractBufferedFile) - I'm able to import and instantiate it directly. It contains lots of actual code that does things.
Neither is there a typing.Protocol that subclasses could conform to that could play the same role. (In fact there is no typing at all, which seems like an issue for a project whose purpose is to ensure interface compatibility. Wouldn't it help solve issues like https://github.com/fsspec/filesystem_spec/issues/1411?)
Finally I can't find a written spec document either, at least not one that conforms to RFC 2119 like the Zarr Spec does for example. The fsspec documentation just points me to fsspec/spec.py.
As a new user / potential developer using fsspec, how am I supposed to know which parts of AbstractFileSystem are implementation details that can be overridden, and which are actual requirements? Is there some reason why this lack of a distinct specification doesn't cause problems in practice? Apologies if I have missed something here, I am new to this whole area.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fsspec/spec.py and the documentation section on the package structure to compare the described AbstractFileSystem and AbstractBufferedFile roles. Review the linked typing issue #625 and compatibility issue #1411 for related context. Done would require a clear decision about the interface contract and documentation of which members are requirements versus implementation details.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100