InvalidCredsError with upload_object_via_stream() on Aurora Objects
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 931
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 4
Description
## Summary
When trying to upload a file to Aurora Objects through `upload_object_via_stream()`, I get an `InvalidCredsError`.
## Detailed Information
When I try the following code:
```python3
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver
cls = get_driver(Provider.AURORAOBJECTS)
driver = cls('api-key', 'secret-key')
container = driver.get_container('container')
container.upload_object_via_stream(iter([b'test']), 'test.txt')
```
It results in the following error:
```
Traceback (most recent call last):
File "", line 1, in
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/base.py", line 216, in upload_object_via_stream
return self.driver.upload_object_via_stream(
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 747, in upload_object_via_stream
return self._put_object_multipart(container=container,
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 988, in _put_object_multipart
upload_id = self._initiate_multipart(container, object_name,
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 572, in _initiate_multipart
response = self.connection.request(request_path, method='POST',
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/common/base.py", line 667, in request
response = responseCls(**kwargs)
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/common/base.py", line 167, in __init__
message=self.parse_error(),
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 138, in parse_error
raise InvalidCredsError(self.body)
libcloud.common.types.InvalidCredsError: 'SignatureDoesNotMatchtx00000000000004e0ebfe5-00620a68e9-1d4a885a-ams021d4a885a-ams02-eu'
```
However `upload_object()` is working fine as follows:
```python3
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver
cls = get_driver(Provider.AURORAOBJECTS)
driver = cls('api-key', 'secret-key')
container = driver.get_container('container')
container.upload_object('test.txt', 'test.txt')
```
I am using Python 3.9.8-1 on Debian unstable with Libcloud 3.4.1.
Contributor guide
Research direction
Start in libcloud/storage/drivers/s3.py at upload_object_via_stream(), _put_object_multipart(), and _initiate_multipart(), comparing that path with the working upload_object() path. Reproduce the Aurora Objects example and trace the multipart initiation request; done means streaming uploads succeed without SignatureDoesNotMatch or InvalidCredsError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100