UpdateDomainName retries too quickly
- Dominant language
- Python
- Stars
- 11.1k
- Forks
- 1k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 2
Description
During recent Chalice deployments, the following error occurred:
```
chalice.deploy.deployer.ChaliceDeploymentError: ERROR - While deploying your chalice application, received the following error:
An error occurred (TooManyRequestsException) when calling the UpdateDomainName
operation (reached max retries: 2): Too Many Requests
```
It appears that Chalice includes this error in the retryable errors for the UpdateDomainName action: https://github.com/aws/chalice/blob/master/chalice/awsclient.py#L815
However, it uses the default 5 second retry period: https://github.com/aws/chalice/blob/master/chalice/awsclient.py#L2044
This exceeds the rate limit for UpdateDomainName actions per account, which is 1 per 30 seconds per account: https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#api-gateway-control-service-limits-table
Additionally, exponential backoff is not implemented for these retries.
Increasing the wait time for retrying and exponential backoff may prevent this issue.
Cheers!
Contributor guide
Research direction
Start in chalice/awsclient.py at the retryable UpdateDomainName handling near line 815 and the retry delay near line 2044. Reproduce or trace the reported Chalice deployment retry path, then verify that retries wait long enough for the API Gateway account limit and use exponential backoff without exceeding the intended retry count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- api, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100