Retry decorator on RateLimitExceeded issue
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 931
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 4
Description
Summary
Enabling retry logic does not work for rate limit exceeded responses.
Detailed Information
retry decorator expects a RateLimitExceeded error to be raised, though session.request method does not raise an exception if a response is returned by the server no matter the status code.
I've tested it on azure_arm driver by setting RETRY_FAILED_HTTP_REQUESTS to True and trying:
while True:
nodes = driver.list_nodes()
print(nodes)
Hitting the throttling limit results in:
Traceback (most recent call last):
File "test.py", line 23, in <module>
nodes = driver.list_nodes()
File "/home/dimgal/libcloud/libcloud/compute/drivers/azure_arm.py", line 395, in list_nodes
r = self.connection.request(action,
File "/home/dimgal/libcloud/libcloud/common/azure_arm.py", line 227, in request
return super(AzureResourceManagementConnection, self) \
File "/home/dimgal/libcloud/libcloud/common/base.py", line 655, in request
response = responseCls(**kwargs)
File "/home/dimgal/libcloud/libcloud/common/base.py", line 165, in __init__
raise exception_from_message(code=self.status,
libcloud.common.exceptions.RateLimitReachedError: 429 Rate limit exceeded
urllib3 supports retries based on HTTP status codes, maybe use this? Let me know how you think this should be handled
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the retry decorator and the request flow in libcloud/common/base.py, then compare the Azure ARM path in libcloud/common/azure_arm.py with the azure_arm driver call shown in the report. Reproduce the configured rate-limit case and determine how a returned 429 becomes RateLimitReachedError. Done means retry behavior works for rate-limit responses when retry logic is enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100