Azure / Azure/azure-sdk-for-rust

Gate inherently cross-partition query shapes on a bounded result set, with an opt-out

Closed
#5,122 0 comments 1 reaction 1 assignee Claimed by @tvaron3 View on GitHub
Client Cosmos
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
112

Description

Raised by @FabianMeiswinkel in review of #5026.

Vector search already sets the precedent: `ORDER BY VectorDistance(...)` requires a `TOP`/`LIMIT` because unbounded execution is pathological. The same argument applies to other inherently cross-partition shapes, which are expensive enough that they're problematic regardless of client memory.

> My 2 cents: I would consider also requesteing a SELECT TOP x (with x <= 1000 or whatever the value in vector search is) - this is a trade-off - it could increase magration problems - but on the other hand these queries are so horribly inefficient that they are problematic anyway.

## Client-side state today

Production advertises `Distinct,MultipleOrderBy,OffsetAndLimit,OrderBy,Top`.

| Shape | Client state | Unbounded? |
| --- | --- | --- |
| Unordered `DISTINCT` | `HashSet` | **Yes** — ~16 B per distinct value |
| Ordered `DISTINCT` | one hash | No — O(1) |
| Cross-partition `ORDER BY` (streaming) | one page per range (`streaming_ordered_merge.rs:218`) | No — O(partitions × page size) |
| `OFFSET`/`LIMIT`/`TOP` | counters | No memory concern, but `OFFSET` fetches-and-discards across every partition (RU cost) |

Note that streaming `ORDER BY` is *not* in the same class as vector search: the backend returns each partition pre-sorted and the client only k-way merges, so peak memory is independent of result-set size. Vector search must buffer globally because ranking isn't decomposable per-partition.

## Not yet implemented — same gate needed when added

| Shape | Client state | Unbounded? |
| --- | --- | --- |
| `GROUP BY` | group key → accumulator map | **Yes** — one entry per distinct group |
| `DCount` | distinct hash set | **Yes** |
| `NonStreamingOrderBy` | full result buffer | **Yes** |
| `HybridSearch` / `WeightedRankFusion` | global ranking buffer | **Yes** |
| Aggregates (`SUM`/`COUNT`/`AVG`/`MIN`/`MAX`) | single accumulator | No — O(1) |

It's worth separating **unbounded client memory** (a real correctness/OOM risk) from **RU cost** (expensive but bounded) — the two may warrant different treatment, and only the first is a hard failure mode.

## Proposed opt-out

Customers need an escape hatch. Straw man:

```rust
QueryOptions {
/// Permits query shapes whose client-side state is not bounded by the
/// query itself (e.g. unordered `DISTINCT` over a high-cardinality path).
allow_unbounded_cross_partition_query: Option, // default: off
}
```

Open design points:

- Per-request (`QueryOptions`) only, or also per-client with per-request override?
- Error shape when the gate trips — a new sub-status in the `20100–20149` input-validation range, with a message naming the offending shape and the remedy.

## Open questions

1. Which shapes gate on **memory** vs. on **RU cost**? Should the two produce different errors?
2. What cap — reuse the vector-search value, or per-shape limits?
3. **Cross-SDK alignment.** .NET and Java accept these queries today. If only Rust rejects them, we create exactly the migration hazard @FabianMeiswinkel flagged. This needs agreement across SDKs before shipping.
4. Should the gate apply when a query is scoped to a single logical partition? Today those are short-circuited to a trivial pipeline before query planning, so no client-side pipeline runs and no unbounded state accumulates.

## Current behavior for reference

Unordered `DISTINCT` is unbounded in Rust, .NET, and Java alike. Rust and .NET both refuse to mint a continuation token for it (Rust: `400 / 20124 ClientDistinctContinuationUnsupported`). Java emits a token that discards the accumulated hash set on resume and silently re-emits duplicates.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.