Classify Bedrock throttles as retryable rate limits with exponential backoff
Open
@Rakshit0306 is already working on this.
Since Sep 10, 2026.
good first issue
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Problem
AWS Bedrock throttling responses such as ThrottlingException and “Too many tokens, please wait before trying again” are currently being mapped as context-window errors. This sends execution into context recovery instead of retrying the request.
Expected behavior
Treat Bedrock throttling as a retryable rate-limit error and retry with exponential backoff, consistent with the provider's throttling semantics.
The retry behavior should:
- Detect the relevant Bedrock throttling responses.
- Avoid triggering context-length recovery for throttling.
- Apply bounded exponential backoff between attempts.
- Preserve the existing behavior for genuine context-window errors.
- Include unit tests covering throttling and non-throttling error classification.
Related Linear ticket: OSS-169
This issue is intentionally separate from guarding the context-recovery path when llm.acall() is unavailable (OSS-170).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.