Azure / Azure/azure-sdk-for-rust

list_blobs (and other XML pagers) deserialize each page body twice

Open
#4,658 1 comment 2 reactions 2 assignees Claimed by @heaths View on GitHub
Azure.Core bug Client
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
109

Description

Copilot raised this issue when I was prompting it about the `list_blobs` perf test.

## Summary
The generated paging callback for XML list operations deserializes the full response body once just to read the continuation marker, then the body is deserialized a **second** time when the caller iterates items or calls `into_model()`. For large pages this doubles XML parsing work and is measurable in perf testing.

## Where it happens

In the generated `list_blobs` callback (`sdk/storage/azure_storage_blob/src/generated/clients/blob_container_client.rs`), the body is parsed to `ListBlobsResponse` only to extract `next_marker`, then the **raw bytes** are stored back into the response:

```rust
let (status, headers, body) = rsp.deconstruct();
let res: ListBlobsResponse = xml::from_xml(&body)?; // (1) full parse, only next_marker used
let rsp = RawResponse::from_bytes(status, headers, body).into(); // raw bytes re-wrapped
Ok(match res.next_marker {
Some(next_marker) if !next_marker.is_empty() => PagerResult::More {
response: rsp,
continuation: PagerContinuation::Token(next_marker),
},
_ => PagerResult::Done { response: rsp },
})
```

Then, when the consumer pulls items, the `Pager`/`ItemIterator` parses the same bytes again via the blanket `Page` impl for `Response` in `azure_core`'s `pager.rs`:

```rust
async fn into_items(self) -> crate::Result {
let page: P = self.into_model()?; // (2) second full parse of the same body
page.into_items().await
}
```

The same applies to explicit page iteration, where the caller calls `into_model()` themselves:

```rust
let mut pager = client.list_blobs(None)?.into_pages();
while let Some(page) = pager.try_next().await? {
let page = page.into_model()?; // (2) second full parse
// ...
}
```

So the body is parsed at (1) and again at (2) — twice per page.

## Impact
- ~2x XML deserialization cost per page for list operations.
- Most visible with large page sizes / many items (e.g., `list_blobs`, `list_containers`, `list_blobs_hierarchical`, and likely other XML pagers).
- Affects both item iteration and `into_pages()` + `into_model()`.

## Possible fixes
1. **Extract only the marker without a full deserialization** in the callback — e.g., parse just the `NextMarker` element, or read it from a header where available, instead of `xml::from_xml::(&body)`.
2. **Carry the already-deserialized model forward** so `into_model()` is a no-op — i.e., have the pager keep the typed `ListBlobsResponse` from step (1) rather than re-wrapping raw bytes, avoiding the second parse.

Either approach removes the redundant parse. Option 2 generalizes across all generated XML pagers if the emitter can stash the parsed model instead of raw bytes.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.