Deletion of S3 objects fails for object paths starting with a slash
- 主要语言
- Python
- 星标
- 2.1k
- 派生
- 931
- 平均合并
- 1 天 2 小时
- 30 天内合并 PR
- 4
描述
## Summary
When trying to delete all objects in an S3 bucket, the deletion fails for objects with a leading slash in their path.
The `delete_object` method returns `True` for these objects, even though they are not deleted.
## Detailed Information
To delete an S3 bucket we use `Container.list_objects()` and `delete_object` to first delete all objects in the bucket. This doesn't work for objects, where the object path starts with a slash. The `delete_object` function returns `True` for these objects even though they are not deleted because the DELETE request is sent to the wrong URL.
For example to delete the object `s3://container-name//path/to/object` the DELETE request is sent to
https://s3-eu-west-1.amazonaws.com/container-name/path/to/object
but it should be sent to this path instead:
https://s3-eu-west-1.amazonaws.com//container-name/path/to/object
The problem is that double slashes are replaced in the request URL in the method `Connection.morph_action_hook`. This is a similar issue to #1529.
To reproduce the issue, you can use the following code (replace credentials and bucket name):
```python
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver
driver = get_driver(Provider.S3)
client = driver("key", "secret", region="eu-west-1")
container = client.get_container("container-name")
objects = container.list_objects()
for obj in objects:
client.delete_object(obj)
client.delete_container(container)
```
This will result in an `ContainerIsNotEmptyError` if the bucket "container-name" contains objects where the object path starts with a slash.
Libcloud version: 3.4.1
Python version: 3.7.12
贡献指南
调研方向
从 Connection.morph_action_hook 开始,使用以斜杠开头的对象路径,通过 S3 示例重现该问题。跟踪 DELETE 请求 URL 的规范化过程;当开头的斜杠得到保留、对象确实被删除,并且随后可以删除 bucket 时,即表示完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- aws, python
- 领域
- cloud
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 描述清楚
- 新手友好度
- 42/100