Azure / Azure/azure-sdk-for-rust

Consider updating `AsyncResponseBody::collect` to use the `content-length` header as a hint for the allocation needed.

Open
#3,893 0 comments 0 reactions 0 assignees View on GitHub
Azure.Core Client
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
112

Description

>AsyncResponseBody::collect now buffers every chunk into Vec and then copies again into a BytesMut. This increases peak memory usage (you retain all chunks and allocate the final buffer) and does an extra pass over the data. To keep memory bounded while still reducing reallocations, consider appending directly into a BytesMut and using reserve(bytes.len()) per chunk (or leveraging a known Content-Length when available) rather than storing all chunks first.

> Copilot might have a good suggestion here, but I'm also wondering: does Storage send a `content-length` header for chunked data so we get a final size? Might not hurt to check, or maybe that's more of a case where they just use `collect_into()`. Still, might not hurt to check if it's there. We could prealloc a `BytesMut`.

_Originally posted by @heaths in [#3879](https://github.com/Azure/azure-sdk-for-rust/pull/3879/changes#r2908016848)_

Contributor guide

Open the contributing guide

Research direction

Start at AsyncResponseBody::collect and compare its buffering path with collect_into, then check whether Storage responses expose a usable content-length for this case. Confirm the chosen approach avoids retaining all chunks while preserving collected response behavior, using relevant repository tests if available.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, performance
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.