awslabs / awslabs/aws-sdk-rust

Allow s3 `ByteStream::read_from` for `futures::Stream<Item = Result<Bytes, E>> + Send`

Open
#1,283 2 comments 1 reaction 0 assignees View on GitHub
feature-request
Dominant language
Rust
Stars
3.3k
Forks
290
Avg merge
1d 12h
Merged PRs (30d)
3

Description

### Describe the feature

Much like bytestream can be constructed from a tokio file, we should also allow it to be created from existing in-memory streams.

I've browsed source to find implementation similar, but couldn't find such so please let me know if an option exists.

### Use Case

I take in multipart file uploads, and have a case where I'd prefer to just pass my existing multipart stream to the client instead of storing in an intermediate file.

### Proposed Solution

An alternative approach to the `FsBuilder` is to have a more generic `IoBuilder` that can take in any type of `impl AsyncRead`

### Other Information

_No response_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### A note for the community

### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue, please leave a comment

Contributor guide

Open the contributing guide

Research direction

Locate the S3 ByteStream implementation and the existing FsBuilder or tokio-file construction path. Compare how those paths represent asynchronous input, then assess the proposed generic IoBuilder and futures::Stream> API. Done means the design is agreed, in-memory streams can construct a ByteStream, and coverage verifies the supported input and error behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.