apache / apache/libcloud

InvalidCredsError with upload_object_via_stream() on Aurora Objects

Open
#1,657 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.1k
Forks
931
Avg merge
1d 2h
Merged PRs (30d)
4

Description

## Summary

When trying to upload a file to Aurora Objects through `upload_object_via_stream()`, I get an `InvalidCredsError`.

## Detailed Information

When I try the following code:

```python3
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver

cls = get_driver(Provider.AURORAOBJECTS)
driver = cls('api-key', 'secret-key')

container = driver.get_container('container')

container.upload_object_via_stream(iter([b'test']), 'test.txt')
```

It results in the following error:

```
Traceback (most recent call last):
File "", line 1, in
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/base.py", line 216, in upload_object_via_stream
return self.driver.upload_object_via_stream(
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 747, in upload_object_via_stream
return self._put_object_multipart(container=container,
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 988, in _put_object_multipart
upload_id = self._initiate_multipart(container, object_name,
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 572, in _initiate_multipart
response = self.connection.request(request_path, method='POST',
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/common/base.py", line 667, in request
response = responseCls(**kwargs)
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/common/base.py", line 167, in __init__
message=self.parse_error(),
File "/home/wout/project/.venv/lib/python3.9/site-packages/libcloud/storage/drivers/s3.py", line 138, in parse_error
raise InvalidCredsError(self.body)
libcloud.common.types.InvalidCredsError: 'SignatureDoesNotMatchtx00000000000004e0ebfe5-00620a68e9-1d4a885a-ams021d4a885a-ams02-eu'
```

However `upload_object()` is working fine as follows:

```python3
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver

cls = get_driver(Provider.AURORAOBJECTS)
driver = cls('api-key', 'secret-key')

container = driver.get_container('container')

container.upload_object('test.txt', 'test.txt')
```

I am using Python 3.9.8-1 on Debian unstable with Libcloud 3.4.1.

Contributor guide

Open the contributing guide

Research direction

Start in libcloud/storage/drivers/s3.py at upload_object_via_stream(), _put_object_multipart(), and _initiate_multipart(), comparing that path with the working upload_object() path. Reproduce the Aurora Objects example and trace the multipart initiation request; done means streaming uploads succeed without SignatureDoesNotMatch or InvalidCredsError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.