awslabs / awslabs/aws-sdk-rust
Allow s3 `ByteStream::read_from` for `futures::Stream<Item = Result<Bytes, E>> + Send`
- Dominant language
- Rust
- Stars
- 3.3k
- Forks
- 290
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 3
Description
### Describe the feature
Much like bytestream can be constructed from a tokio file, we should also allow it to be created from existing in-memory streams.
I've browsed source to find implementation similar, but couldn't find such so please let me know if an option exists.
### Use Case
I take in multipart file uploads, and have a case where I'd prefer to just pass my existing multipart stream to the client instead of storing in an intermediate file.
### Proposed Solution
An alternative approach to the `FsBuilder` is to have a more generic `IoBuilder` that can take in any type of `impl AsyncRead`
### Other Information
_No response_
### Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### A note for the community
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue, please leave a comment
Contributor guide
Research direction
Locate the S3 ByteStream implementation and the existing FsBuilder or tokio-file construction path. Compare how those paths represent asynchronous input, then assess the proposed generic IoBuilder and futures::Stream> API. Done means the design is agreed, in-memory streams can construct a ByteStream, and coverage verifies the supported input and error behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100