Azure / Azure/azure-sdk-for-rust
Cosmos: Reconcile Dataflow Pipeline with PK Range ID resolution
- Dominant language
- Rust
- Stars
- 884
- Forks
- 365
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 109
Description
The Dataflow pipeline calls in to the routing map to resolve the PK Range ID for most operations (basically, unless they have a Logical Partition Key, because then we know it maps to one physical partition and the gateway has the latest routing map so it's better to defer to it). Given that, I think we can clean up the logic in `pre_resolve_partition_key_range_id`. That logic runs in the operation pipeline and any operation that _needs_ a PK Range ID should already have one from the dataflow pipeline.
As part of this, we should extend the dataflow pipeline to _also_ resolve the PK Range ID for single-partition operations (where we have a logical partition key). We don't do that today because we don't need to do it for routing, but it's still useful for PPCB.
Contributor guide
Research direction
Start by locating `pre_resolve_partition_key_range_id` and the Dataflow pipeline entry points, then trace how operations with and without a logical partition key obtain the PK Range ID. Done means single-partition operations are resolved in the Dataflow pipeline and the operation pipeline no longer performs redundant resolution while routing behavior remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, rust
- Domain
- databases, distributed-systems
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100