ISISNeutronMuon / ISISNeutronMuon/analytics-data-platform

Remove pin for s3fs < 2026.2.0

Open
#237 0 comments 0 reactions 0 assignees View on GitHub
technical debt
Dominant language
Python
Stars
0
Forks
0
Avg merge
2d 7h
Merged PRs (30d)
21

Description

The extract and load tests fail when upgrading s3fs past 2026.1.0, e.g

```bash
> cd infra/local
> docker compose up -d
> cd warehouses/accelerator_ops/extract_load/opralogweb/
> pytest --catalog-type=rest tests/
```

```python
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 2587, in commit
write_result = self._call_s3("put_object", **kw, **match)
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 2421, in _call_s3
return self.fs.call_s3(method, self.s3_additional_kwargs, *kwarglist, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/fsspec/asyn.py", line 103, in sync
raise return_result
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/fsspec/asyn.py", line 56, in _runner
result[0] = await coro
^^^^^^^^^^
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 451, in _call_s3
return await _error_wrapper(
^^^^^^^^^^^^^^^^^^^^^
method, kwargs=additional_kwargs, retries=self.retries
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 214, in _error_wrapper
raise err
PermissionError: Access Denied.
```

The issues seems to have arisen when fsspec/s3fs#1002 was merged. Reversing the changes locally recovers the original behaviour.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.