Azure / Azure/azure-sdk-for-rust

[Storage] BlobContainerClient::list_blobs is missing delimiter support — hierarchical listing impossible

Open
#4,455 3 comments 0 reactions 1 assignee Claimed by @vincenttran-msft View on GitHub
Client customer-reported needs-team-attention Storage
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
109

Description

The GA `azure_storage_blob 1.0` rewrite dropped `delimiter` from `list_blobs`, leaving filesystem-shaped consumers unable to use the SDK for any non-recursive listing. The gap appears in three places, each of which closes off a different workaround:

### 1. `BlobContainerClientListBlobsOptions` has no `delimiter` field

[`generated/models/method_options.rs`](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/storage/azure_storage_blob/src/generated/models/method_options.rs) — the options struct exposes `include`, `marker`, `maxresults`, `prefix`, `start_from`, `timeout`, but not `delimiter`. The underlying [REST API supports it](https://learn.microsoft.com/en-us/rest/api/storageservices/list-blobs#uri-parameters), and every other Azure SDK language ships hierarchical listing in their GA crates:

- .NET: `BlobContainerClient.GetBlobsByHierarchy`
- Java: `BlobContainerClient.listBlobsByHierarchy`
- Python: `ContainerClient.walk_blobs`
- JavaScript: `ContainerClient.listBlobsByHierarchy`
- Go: `Client.NewListBlobsHierarchyPager`

### 2. `ListBlobsResponse` has no field for `` children

[`generated/models/models.rs`](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/storage/azure_storage_blob/src/generated/models/models.rs) — even if a caller could inject `delimiter` somehow, the response model has no place to put the `` entries the service returns.

### 3. The `` deserializer silently drops `` children

[`generated/models/xml_helpers.rs`](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/storage/azure_storage_blob/src/generated/models/xml_helpers.rs):

```rust
pub(crate) struct Blob_itemsBlobItem {
#[serde(default)]
Blob: Vec,
}
```

Only `` is captured; `` siblings are silently discarded by serde. A custom-type override (per the `@alternateType` discussion in #3258) wouldn't help here — the data isn't reaching userland.

### 4. Pipeline injection is blocked

`BlobServiceClient.pipeline` and `BlobContainerClient.pipeline` are `pub(crate)`, so consumers can't build a `Request` with `&delimiter=/` and send it through the SDK's auth/retry stack manually. The only path is re-implementing the whole HTTP+auth layer with `reqwest`.

## Concrete impact

Consumers who need hierarchical listing today must choose between:

- **Client-side grouping** after a flat `list_blobs`. Correct but pulls every blob under the prefix; O(N) instead of O(page) for ""show me the immediate children of `/docs/`"". On a container with a million blobs under `/docs/`, this is the difference between a single API call and several thousand.
- **Bypassing the SDK** with raw HTTP. Loses the `BearerTokenAuthorizationPolicy` / retry / tracing pipeline. We ended up doing this in our backend port — re-implementing auth-and-send against `reqwest` just to add one query parameter.

The beta `azure_storage_blobs 0.21.x` had hierarchical listing working (see closed #175). It's a regression in the 1.0 cycle.

## Suggested fix

1. Add `delimiter: Option` to `BlobContainerClientListBlobsOptions`.
2. Add a typed `BlobPrefix { name: BlobName }` model.
3. Add `blob_prefixes: Vec` to `ListBlobsResponse`, with a `Blob_prefixesBlobPrefix` XML-helper mirroring `Blob_itemsBlobItem`.

The underlying TypeSpec spec has a `ContainersListBlobHierarchySegmentOptions` shape — this looks like the emitter chose `ListBlobFlatSegment` over `ListBlobHierarchySegment` when generating the convenience layer.

## Repro / version

- `azure_storage_blob = "1.0.0"` (GA, current latest on crates.io)
- Hit while porting an Azure backend plugin from a hand-rolled `reqwest`+HMAC layer onto the GA SDK; non-recursive `list` against multi-tier containers was a blocker.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.