fsspec / fsspec/filesystem_spec
lpath of filesystem.get doesn't handle pathlib.Path gracefully
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
I didn't find an existing issue, so I am opening a new one.
It looks like we can't pass pathlib Path's as local paths for get. I've only tried it against the GitHub implementation so far but considering that it fails in what appears to be a general utility function it might affect other implementations, too. Here is a reproducing example of the problem:
>>> import fsspec
>>> from pathlib import Path
>>> foo = Path("test_file.md")
>>> fs = fsspec.filesystem("github", org="fsspec", repo="filesystem_spec")
>>> fs.get("README.md", foo)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Sebastian\AppData\Local\Programs\Python\Python38\lib\site-packages\fsspec\spec.py", line 771, in get
lpaths = other_paths(rpaths, lpath)
File "C:\Users\Sebastian\AppData\Local\Programs\Python\Python38\lib\site-packages\fsspec\utils.py", line 380, in other_paths
assert len(paths) == len(path2)
TypeError: object of type 'WindowsPath' has no len()
>>> fs.get("README.md", str(foo)) # this works fine
>>>
Would it be possible to allow inputs of type pathlib.path for lpath? (e.g., by simply converting them to posix path strings: lpath.as_posix())
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with get in fsspec/spec.py and the other_paths utility in fsspec/utils.py, using the pathlib.Path reproduction from the issue. Verify that a Path can be used as lpath while the existing string input continues to work; completion is demonstrated when the reproduction no longer raises the WindowsPath len() TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100