Azure / Azure/azure-sdk-for-rust
[Epic] Cosmos: Advanced Cross-Partition Queries
- Dominant language
- Rust
- Stars
- 884
- Forks
- 365
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 109
Description
### Feature Summary
Support advanced cross-partition query execution in the production `sdk/cosmos/azure_data_cosmos_driver` query pipeline.
### Feature Description
`planner::validate_query_plan` currently rejects ORDER BY, aggregate, GROUP BY, DISTINCT, TOP/LIMIT, and hybrid plans. Existing parser and evaluator support does not provide production cross-partition execution for these plan shapes.
Validation and acceptance criteria:
- Compose advanced stages through a production pipeline factory rather than one-off execution paths.
- Apply Gateway rewritten queries to partition-level requests.
- Introduce internal result envelopes covering `payload`, `orderByItems`, `groupByItems`, aggregate shapes, and score/statistics shapes.
- Transform internal envelopes into public responses while preserving response metadata and diagnostics.
- Define and enforce buffering/resource limits for stages that drain or retain cross-partition state.
- Define an explicit continuation policy for every stage and reject unsupported external continuations rather than returning unsafe tokens.
- Add reusable dataflow tests for stage composition, pagination, failures, partition splits, and response transformation.
- Preserve cross-SDK composition order: merge/ORDER BY -> aggregate -> DISTINCT -> GROUP BY -> OFFSET -> LIMIT -> TOP -> DCOUNT -> empty-page suppression.
- Make streaming ORDER BY resumable and split-safe.
- Treat aggregate, GROUP BY, buffered ORDER BY, and hybrid execution as initially non-resumable.
### Use Case
Applications need the Rust Cosmos DB driver to execute advanced SQL queries correctly across physical partitions with production pagination, diagnostics, deterministic ordering, and explicit resource and continuation behavior.
### Alternatives
Relying on parser/evaluator support or client-side application aggregation is insufficient because neither supplies the production partition fan-out, merge semantics, continuation safety, split handling, or response metadata guarantees required by the driver.
### Additional Context
The implementation belongs in `sdk/cosmos/azure_data_cosmos_driver/src/driver/dataflow/`. Shared prerequisites live only in this parent and are dependencies for all child issues; no separate shared-foundation issue is needed.
Contributor guide
Assessment
This issue has not been assessed yet.