awslabs / awslabs/aws-sdk-rust
S3 - Delete all files under a folder (recursive delete)
- Dominant language
- Rust
- Stars
- 3.3k
- Forks
- 290
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 3
Description
### Describe the feature
Delete all files underneath a folder (recursive delete) using a prefix.
### Use Case
It would be nice to be able to give the prefix and delete under it versus trying to iterate through all the objects and deleting them individually.
ex. Bucket structure
- folder1
- file 1
- file 2
- file 3
- folder2
- file 4
- folder3
Delete everything under "folder1/"
### Proposed Solution
Ideally it would be nice if delete_objects took the prefix builder function argument.
ex.
```rust
// Deletes all files in folder1
let bucket = "my-bucket";
let prefix = "folder1/";
let s3res = s3.delete_objects()
.bucket(bucket)
.prefix(prefix)
.send();
```
A possible workaround for right now might be, iterating through all the objects then building the delete_objects vec using the iterated page keys. Though I'm not sure if there's any gotchas or issues with this approach.
```rust
use std::error::Error;
use tokio_stream::StreamExt;
#[tokio::main]
async fn main() -> Result<(), Box> {
let shared_config = aws_config::load_from_env().await;
let s3 = aws_sdk_s3::Client::new(&shared_config);
let prefix = "prefix";
let bucket = "bucket";
let mut pages = s3.list_objects_v2()
.bucket(bucket)
.prefix(prefix)
.into_paginator()
.send();
let mut delete_objects: Vec = vec![];
while let Some(page) = pages.next().await {
let obj_id = ObjectIdentifier::builder().set_key(Some(page?.key)).build();
delete_objects.push(obj_id);
}
let delete = Delete::builder().set_objects(Some(delete_objects)).build();
s3.delete_objects()
.bucket(bucket)
.delete(delete)
.send()
.await?;
println!("Objects deleted.");
Ok(())
}
```
### Other Information
Possible temporary workaround provided in proposed solution.
### Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### A note for the community
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue, please leave a comment
Contributor guide
Research direction
Start with the Rust S3 delete_objects and list_objects_v2 paginator APIs shown in the issue, then verify how Delete and ObjectIdentifier are modeled. Compare the requested prefix behavior with the existing pagination workaround and determine the required API design. Done means an agreed implementation path is established, with tests or documentation covering recursive deletion under a prefix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100