fsspec / fsspec/filesystem_spec
Use of '::' in file's name
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
I am investigating this issue in uproot, where we try to open a file that contains :: in the name of the file. As you can see, the expected behavior would be to correctly create the file indicated in the string, but it doesn't seem to be the case. The reason is in this function
https://github.com/fsspec/filesystem_spec/blob/fe59f48363029a0da68a0de534bf63e1c42e5a81/fsspec/core.py#L331-L366
where, it seems to me, the case of :: being part of the file name is not considered, and it is only treated as a protocol separator.
One idea to adapt the code would be the following:
if "::" in path:
x = re.compile(".*[^a-z]+.*") # test for non protocol-like single word
bits = []
for p in path.split("::"):
# Check if part looks like a protocol or URL
if "://" in p or x.match(p) or p in known_implementations:
bits.append(p)
else:
# If not, assume it is part of the file name
bits.append(p + "://")
# If no part matches a known protocol, treat the entire path as a file name
if not any(b for b in bits if b.strip("://") in known_implementations):
bits = [path]
else:
bits = [path]
This fixes Jim's reproducer, but breaks a few tests, making me wonder if this behavior is intentional.
The question is thus the following: should a logic be implemented in fsspec to handle the case in which :: is part of the file name or should we implement a check in uproot where we raise an error if :: is not used as a protocol separator?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fsspec/core.py lines 331-366 and the linked uproot issue 1251, then reproduce the filename containing ::. Run the affected tests, including the cases that the proposed parsing change breaks. Done requires a decided behavior for :: and regression coverage for both filenames and protocol separators.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100