Azure / Azure/azure-sdk-for-rust

Cosmos: Reconcile Dataflow Pipeline with PK Range ID resolution

Open
#4,629 0 comments 1 reaction 0 assignees View on GitHub
Cosmos
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
109

Description

The Dataflow pipeline calls in to the routing map to resolve the PK Range ID for most operations (basically, unless they have a Logical Partition Key, because then we know it maps to one physical partition and the gateway has the latest routing map so it's better to defer to it). Given that, I think we can clean up the logic in `pre_resolve_partition_key_range_id`. That logic runs in the operation pipeline and any operation that _needs_ a PK Range ID should already have one from the dataflow pipeline.

As part of this, we should extend the dataflow pipeline to _also_ resolve the PK Range ID for single-partition operations (where we have a logical partition key). We don't do that today because we don't need to do it for routing, but it's still useful for PPCB.

Contributor guide

Open the contributing guide

Research direction

Start by locating `pre_resolve_partition_key_range_id` and the Dataflow pipeline entry points, then trace how operations with and without a logical partition key obtain the PK Range ID. Done means single-partition operations are resolved in the Dataflow pipeline and the operation pipeline no longer performs redundant resolution while routing behavior remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, rust
Domain
databases, distributed-systems
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.