Azure / Azure/azure-sdk-for-python

[Cosmos] Partition key range cache: resume from last failed page and move fetch to background

Open
#47,026 0 comments 0 reactions 1 assignee Claimed by @dibahlfi View on GitHub
Client Cosmos
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

## Summary

The partition key range cache in `azure-cosmos` has two improvement opportunities:

### 1. Resume from last failed page instead of restarting

In `_fetch_routing_map` (sync: `routing_map_provider.py`, async: `aio/routing_map_provider.py`), when `_ReadPartitionKeyRanges` raises a `CosmosHttpResponseError`, the exception propagates up and the caller (`get_routing_map`) does not persist any partial progress. On the next attempt (force-refresh or cache miss), the fetch restarts from scratch with no continuation/etag, re-fetching pages that previously succeeded.

**Current behavior:**
```python
try:
pk_range_generator = self._document_client._ReadPartitionKeyRanges(...)
ranges.extend(list(pk_range_generator))
except CosmosHttpResponseError as e:
logger.error(...)
raise # all progress lost — next attempt starts from page 1
```

**Proposed behavior:**
- On failure, preserve the etag/continuation from the last successful page in the cache entry.
- On the next cache population attempt, use that continuation to resume rather than starting over.
- This is especially relevant for containers with a large number of physical partitions where the response spans multiple pages.

### 2. Move partition key range fetch to background

Currently, the pkrange fetch is performed **synchronously on the critical path** of the first data-plane request that needs routing information. For large containers or high-latency accounts, this blocks the user's request while the full set of ranges is fetched.

**Proposed behavior:**
- Perform the pkrange cache population in a background task (thread or asyncio task).
- If a request arrives while the background fetch is in progress, either:
- Wait on the in-flight fetch (current single-flight behavior is acceptable here), or
- Return a partial/stale routing map if available while the refresh completes in the background.
- This reduces first-request latency for large containers.

## Affected files

- `sdk/cosmos/azure-cosmos/azure/cosmos/_routing/routing_map_provider.py`
- `sdk/cosmos/azure-cosmos/azure/cosmos/_routing/aio/routing_map_provider.py`
- `sdk/cosmos/azure-cosmos/azure/cosmos/_routing/_routing_map_provider_common.py`

## Sibling issue

- Rust SDK: https://github.com/Azure/azure-sdk-for-rust/issues/4437

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.