fsspec / fsspec/filesystem_spec
Missing **kwarg in open_files when calling mkdirs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
I was trying to write to a bucket directory where I pass as kwargs region_name, and I run into a permission error that can be seen here https://github.com/coiled/coiled-runtime/runs/6682060242?check_suite_focus=true#step:6:177
In fsspec/core.py in open_files we are not passing the kwargs when calling mkdirs, see https://github.com/fsspec/filesystem_spec/blob/06c400335cb7d382cd57d94f93a4b30157dad9b2/fsspec/core.py#L310
This is causing a problem on s3fs when re-creating the bucket in this code https://github.com/fsspec/s3fs/blob/3e9b7d4a62f6efb3d282d9ded2baf4625fab0515/s3fs/core.py#L767-L784 as the parents don't have the kwargs.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at fsspec/core.py open_files around line 310 and compare the mkdirs call with the kwargs available there. Reproduce the linked s3fs permission case using region_name; done when the open_files path preserves those filesystem kwargs for directory creation and the relevant existing tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100