version 2023.3.0 issue
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
After upgrade sf3 and fsspectp 2023.3.0 we see the following error, is there a bug or can someone advise how to resolve this? Thanks.
[ERROR] PermissionError: [Errno 1] The specified method is not allowed against this resource.
Traceback (most recent call last):
File "/var/task/google_analytics/etl.py", line 119, in run_reports
df.to_csv(get_s3_location(report), index=False)
File "/var/task/pandas/core/generic.py", line 3551, in to_csv
return DataFrameRenderer(formatter).to_csv(
File "/var/task/pandas/io/formats/format.py", line 1180, in to_csv
csv_formatter.save()
File "/var/task/pandas/io/formats/csvs.py", line 241, in save
with get_handle(
File "/var/task/pandas/io/common.py", line 667, in get_handle
ioargs = _get_filepath_or_buffer(
File "/var/task/pandas/io/common.py", line 393, in _get_filepath_or_buffer
file_obj = fsspec.open(
File "/var/task/fsspec/core.py", line 419, in open
return open_files(
File "/var/task/fsspec/core.py", line 283, in open_files
[fs.makedirs(parent, exist_ok=True) for parent in parents]
File "/var/task/fsspec/core.py", line 283, in
[fs.makedirs(parent, exist_ok=True) for parent in parents]
File "/var/task/fsspec/asyn.py", line 115, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/var/task/fsspec/asyn.py", line 100, in sync
raise return_result
File "/var/task/fsspec/asyn.py", line 55, in _runner
result[0] = await coro
File "/var/task/s3fs/core.py", line 892, in _makedirs
await self._mkdir(path, create_parents=True)
File "/var/task/s3fs/core.py", line 877, in _mkdir
await self._call_s3("create_bucket", **params)
File "/var/task/s3fs/core.py", line 347, in _call_s3
return await _error_wrapper(
File "/var/task/s3fs/core.py", line 139, in _error_wrapper
raise err
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with google_analytics/etl.py at line 119, then trace the pandas to_csv call through fsspec/core.py and s3fs/core.py, especially the _makedirs and _mkdir paths shown in the traceback. Reproduce the 2023.3.0 failure and determine whether the reported PermissionError is caused by the upgrade; done means the cause and a verified resolution are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, pandas, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100