Azure / Azure/azure-sdk-for-python
Cosmos: Per-Partition Circuit Breaker should track 403:3 (WriteForbidden) errors
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
## Description
The per-partition circuit breaker (PPCB) should track `403:3` (`WriteForbidden`) errors. Currently, PPCB records failures generically without considering `403` status code with sub-status `3`, which indicates a write-region failover scenario in single-writer accounts.
## Context
- `403:3` (`SubStatusCodes.WRITE_FORBIDDEN`) signals that writes are forbidden on the current region, typically during a write-region failover.
- The existing endpoint discovery retry policy already handles `403:3` for region failover ([`_endpoint_discovery_retry_policy.py` L100](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cosmos/azure-cosmos/azure/cosmos/_endpoint_discovery_retry_policy.py#L100)), but PPCB does not currently factor this error into its partition health tracking.
- Incorporating `403:3` into PPCB would allow the circuit breaker to more accurately reflect partition health during region transitions and avoid routing requests to regions that are returning write-forbidden errors.
## Expected Behavior
PPCB should recognize `403:3` as a trackable failure so that partition health state is updated accordingly when a region begins returning `WriteForbidden` errors. This would allow PPCB to proactively route requests away from affected regions/partitions during failover scenarios.
Contributor guide
Assessment
This issue has not been assessed yet.