awslabs / awslabs/aws-sdk-rust

S3 - Delete all files under a folder (recursive delete)

Open
#529 4 comments 3 reactions 0 assignees View on GitHub
feature-request high-level-library p2
Dominant language
Rust
Stars
3.3k
Forks
290
Avg merge
1d 12h
Merged PRs (30d)
3

Description

### Describe the feature

Delete all files underneath a folder (recursive delete) using a prefix.

### Use Case

It would be nice to be able to give the prefix and delete under it versus trying to iterate through all the objects and deleting them individually.

ex. Bucket structure

- folder1
- file 1
- file 2
- file 3
- folder2
- file 4
- folder3

Delete everything under "folder1/"

### Proposed Solution

Ideally it would be nice if delete_objects took the prefix builder function argument.

ex.
```rust
// Deletes all files in folder1
let bucket = "my-bucket";
let prefix = "folder1/";
let s3res = s3.delete_objects()
.bucket(bucket)
.prefix(prefix)
.send();
```

A possible workaround for right now might be, iterating through all the objects then building the delete_objects vec using the iterated page keys. Though I'm not sure if there's any gotchas or issues with this approach.

```rust
use std::error::Error;
use tokio_stream::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box> {
let shared_config = aws_config::load_from_env().await;
let s3 = aws_sdk_s3::Client::new(&shared_config);
let prefix = "prefix";
let bucket = "bucket";
let mut pages = s3.list_objects_v2()
.bucket(bucket)
.prefix(prefix)
.into_paginator()
.send();

let mut delete_objects: Vec = vec![];
while let Some(page) = pages.next().await {
let obj_id = ObjectIdentifier::builder().set_key(Some(page?.key)).build();
delete_objects.push(obj_id);
}

let delete = Delete::builder().set_objects(Some(delete_objects)).build();

s3.delete_objects()
.bucket(bucket)
.delete(delete)
.send()
.await?;

println!("Objects deleted.");

Ok(())
}
```

### Other Information

Possible temporary workaround provided in proposed solution.

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### A note for the community

### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue, please leave a comment

Contributor guide

Open the contributing guide

Research direction

Start with the Rust S3 delete_objects and list_objects_v2 paginator APIs shown in the issue, then verify how Delete and ObjectIdentifier are modeled. Compare the requested prefix behavior with the existing pagination workaround and determine the required API design. Done means an agreed implementation path is established, with tests or documentation covering recursive deletion under a prefix.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, rust
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.