XAmzContentSHA256Mismatch writing to storage bucket
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
When writing to a Hetzner Storage Bucket, which supports same AWS and boto clients, I received an XAmzContentSHA256Mismatch
import s3fs
bucket_client = s3fs.S3FileSystem(
key="***",
secret="***",
endpoint_url="**",
)
with bucket_client.open("s3://mybucker/test.txt", "wb") as f:
f.write(b"eebhdbedbehdbehdbehj")
Logs:
File [~/miniconda/lib/python3.12/site-packages/s3fs/core.py:2290](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/s3fs/core.py#line=2289), in S3File._call_s3(self, method, *kwarglist, **kwargs)
2289 def _call_s3(self, method, *kwarglist, **kwargs):
-> 2290 return self.fs.call_s3(method, self.s3_additional_kwargs, *kwarglist, **kwargs)
File [~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py:118](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py#line=117), in sync_wrapper.<locals>.wrapper(*args, **kwargs)
115 @functools.wraps(func)
116 def wrapper(*args, **kwargs):
117 self = obj or args[0]
--> 118 return sync(self.loop, func, *args, **kwargs)
File [~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py:103](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py#line=102), in sync(loop, func, timeout, *args, **kwargs)
101 raise FSTimeoutError from return_result
102 elif isinstance(return_result, BaseException):
--> 103 raise return_result
104 else:
105 return return_result
File [~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py:56](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py#line=55), in _runner(event, coro, result, timeout)
54 coro = asyncio.wait_for(coro, timeout=timeout)
55 try:
---> 56 result[0] = await coro
57 except Exception as ex:
58 result[0] = ex
File [~/miniconda/lib/python3.12/site-packages/s3fs/core.py:371](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/s3fs/core.py#line=370), in S3FileSystem._call_s3(self, method, *akwarglist, **kwargs)
369 logger.debug("CALL: %s - %s - %s", method.__name__, akwarglist, kw2)
370 additional_kwargs = self._get_s3_method_kwargs(method, *akwarglist, **kwargs)
--> 371 return await _error_wrapper(
372 method, kwargs=additional_kwargs, retries=self.retries
373 )
File [~/miniconda/lib/python3.12/site-packages/s3fs/core.py:146](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/s3fs/core.py#line=145), in _error_wrapper(func, args, kwargs, retries)
144 err = e
145 err = translate_boto_error(err)
--> 146 raise err
OSError: [Errno 5] An error occurred (XAmzContentSHA256Mismatch) when calling the PutObject operation: None
Versions:
- Python 3.12.2
- s3fs==2024.12.0
- aioboto3==13.4.0
- aiobotocore==2.18.0
- boto==2.49.0
- boto3==1.36.1
- botocore==1.36.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the failing PutObject path in s3fs/core.py around lines 371 and 2290, using the provided versions and endpoint. Trace through fsspec/asyn.py around lines 56-118 and reproduce the write; done is a confirmed write without XAmzContentSHA256Mismatch or a documented compatibility limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100