Azure / Azure/azure-sdk-for-python

Cosmos: Per-Partition Circuit Breaker should track 403:3 (WriteForbidden) errors

Open
#46,328 0 comments 0 reactions 0 assignees View on GitHub
Client Cosmos
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

## Description

The per-partition circuit breaker (PPCB) should track `403:3` (`WriteForbidden`) errors. Currently, PPCB records failures generically without considering `403` status code with sub-status `3`, which indicates a write-region failover scenario in single-writer accounts.

## Context

- `403:3` (`SubStatusCodes.WRITE_FORBIDDEN`) signals that writes are forbidden on the current region, typically during a write-region failover.
- The existing endpoint discovery retry policy already handles `403:3` for region failover ([`_endpoint_discovery_retry_policy.py` L100](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cosmos/azure-cosmos/azure/cosmos/_endpoint_discovery_retry_policy.py#L100)), but PPCB does not currently factor this error into its partition health tracking.
- Incorporating `403:3` into PPCB would allow the circuit breaker to more accurately reflect partition health during region transitions and avoid routing requests to regions that are returning write-forbidden errors.

## Expected Behavior

PPCB should recognize `403:3` as a trackable failure so that partition health state is updated accordingly when a region begins returning `WriteForbidden` errors. This would allow PPCB to proactively route requests away from affected regions/partitions during failover scenarios.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.