apache / apache/libcloud

Deletion of S3 objects fails for object paths starting with a slash

オープン
#1,654 コメント 7 件 リアクション 0 件 担当者 0 名 GitHub で見る
api: storage drivers: aws s3
主要言語
Python
スター
2.1k
フォーク
931
平均マージ
1日 2時間
マージ済み PR(30日)
4

説明

## Summary

When trying to delete all objects in an S3 bucket, the deletion fails for objects with a leading slash in their path.
The `delete_object` method returns `True` for these objects, even though they are not deleted.

## Detailed Information

To delete an S3 bucket we use `Container.list_objects()` and `delete_object` to first delete all objects in the bucket. This doesn't work for objects, where the object path starts with a slash. The `delete_object` function returns `True` for these objects even though they are not deleted because the DELETE request is sent to the wrong URL.
For example to delete the object `s3://container-name//path/to/object` the DELETE request is sent to
https://s3-eu-west-1.amazonaws.com/container-name/path/to/object
but it should be sent to this path instead:
https://s3-eu-west-1.amazonaws.com//container-name/path/to/object
The problem is that double slashes are replaced in the request URL in the method `Connection.morph_action_hook`. This is a similar issue to #1529.

To reproduce the issue, you can use the following code (replace credentials and bucket name):
```python
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver

driver = get_driver(Provider.S3)
client = driver("key", "secret", region="eu-west-1")

container = client.get_container("container-name")
objects = container.list_objects()
for obj in objects:
client.delete_object(obj)
client.delete_container(container)
```
This will result in an `ContainerIsNotEmptyError` if the bucket "container-name" contains objects where the object path starts with a slash.

Libcloud version: 3.4.1
Python version: 3.7.12

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Connection.morph_action_hookから始め、スラッシュで始まるオブジェクトパスを使ってS3の例で問題を再現します。DELETEリクエストのURLがどのように正規化されるかを追跡します。先頭のスラッシュが保持され、オブジェクトが実際に削除され、その後バケットを削除できれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
aws, python
領域
cloud
issue の種類
バグ
難易度
3/5
見積もり時間
1〜2日
活発さ
停滞
明瞭さ
明確に書かれている
初心者へのやさしさ
42/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。