Azure / Azure/azure-sdk-for-rust

[Epic] Cosmos: Advanced Cross-Partition Queries

Open
#4,753 0 comments 1 reaction 1 assignee Claimed by @tvaron3 View on GitHub
Client Cosmos Epic
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
109

Description

### Feature Summary

Support advanced cross-partition query execution in the production `sdk/cosmos/azure_data_cosmos_driver` query pipeline.

### Feature Description

`planner::validate_query_plan` currently rejects ORDER BY, aggregate, GROUP BY, DISTINCT, TOP/LIMIT, and hybrid plans. Existing parser and evaluator support does not provide production cross-partition execution for these plan shapes.

Validation and acceptance criteria:

- Compose advanced stages through a production pipeline factory rather than one-off execution paths.
- Apply Gateway rewritten queries to partition-level requests.
- Introduce internal result envelopes covering `payload`, `orderByItems`, `groupByItems`, aggregate shapes, and score/statistics shapes.
- Transform internal envelopes into public responses while preserving response metadata and diagnostics.
- Define and enforce buffering/resource limits for stages that drain or retain cross-partition state.
- Define an explicit continuation policy for every stage and reject unsupported external continuations rather than returning unsafe tokens.
- Add reusable dataflow tests for stage composition, pagination, failures, partition splits, and response transformation.
- Preserve cross-SDK composition order: merge/ORDER BY -> aggregate -> DISTINCT -> GROUP BY -> OFFSET -> LIMIT -> TOP -> DCOUNT -> empty-page suppression.
- Make streaming ORDER BY resumable and split-safe.
- Treat aggregate, GROUP BY, buffered ORDER BY, and hybrid execution as initially non-resumable.

### Use Case

Applications need the Rust Cosmos DB driver to execute advanced SQL queries correctly across physical partitions with production pagination, diagnostics, deterministic ordering, and explicit resource and continuation behavior.

### Alternatives

Relying on parser/evaluator support or client-side application aggregation is insufficient because neither supplies the production partition fan-out, merge semantics, continuation safety, split handling, or response metadata guarantees required by the driver.

### Additional Context

The implementation belongs in `sdk/cosmos/azure_data_cosmos_driver/src/driver/dataflow/`. Shared prerequisites live only in this parent and are dependencies for all child issues; no separate shared-foundation issue is needed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.