Azure / Azure/azure-sdk-for-rust
Consider updating `AsyncResponseBody::collect` to use the `content-length` header as a hint for the allocation needed.
- Dominant language
- Rust
- Stars
- 884
- Forks
- 365
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 112
Description
>AsyncResponseBody::collect now buffers every chunk into Vec and then copies again into a BytesMut. This increases peak memory usage (you retain all chunks and allocate the final buffer) and does an extra pass over the data. To keep memory bounded while still reducing reallocations, consider appending directly into a BytesMut and using reserve(bytes.len()) per chunk (or leveraging a known Content-Length when available) rather than storing all chunks first.
> Copilot might have a good suggestion here, but I'm also wondering: does Storage send a `content-length` header for chunked data so we get a final size? Might not hurt to check, or maybe that's more of a case where they just use `collect_into()`. Still, might not hurt to check if it's there. We could prealloc a `BytesMut`.
_Originally posted by @heaths in [#3879](https://github.com/Azure/azure-sdk-for-rust/pull/3879/changes#r2908016848)_
Contributor guide
Research direction
Start at AsyncResponseBody::collect and compare its buffering path with collect_into, then check whether Storage responses expose a usable content-length for this case. Confirm the chosen approach avoids retaining all chunks while preserving collected response behavior, using relevant repository tests if available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, performance
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100