fsspec / fsspec/filesystem_spec
fsspec URIs with '+' instead of '::'
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
Hi,
I wanted to ask about the fsspec convention of using '::' to chain URL handling protocols (e.g. simplecache with s3fs). Using '::' means the resulting string is not a URI (e.g. 'simplecache::s3fs://bucket_name/myfile') as the 'simplecache::s3fs' violates https://datatracker.ietf.org/doc/html/rfc3986#section-3.1
This is also related to pandas not being able to use these (see https://github.com/pandas-dev/pandas/issues/59684) as it checks if the path given to pandas follows RFC3986. My question is if it would be possible to also allow specification of '+' to chain URL, which would then result in a valid URI.
This would not cover the more complicated chaining cases, but at least the simple one.
Thanks
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No implementation files, entry points, or tests are named. Start by locating fsspec's protocol and URL parsing logic, then compare its current '::' chaining behavior with RFC 3986 and the linked pandas issue. Done should include a defined '+' chaining behavior, compatibility with existing cases, and tests for simple chained URLs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100