Add an actor decorator argument to compute backoff time using a function
- Dominant language
- Python
- Stars
- 5.3k
- Forks
- 383
- Avg merge
- 9h 41m
- Merged PRs (30d)
- 2
Description
## What version of Dramatiq are you using?
1.14.2
## Description
Right now we can provide per-actor settings for the retry policy using a predicate in the actor decorator's argument `retry_when`. However, there is no such option to override backoff times used for retrying tasks.
The case: I am using an actor to send a webhook to an external service. We have agreed that a webhook should be retried three times if the first attempt fails. Second attempt after 1 minute, third attempt after 5 minutes, fourth attempt after an hour. It is not easy to implement such logic in dramatiq at the moment.
Having spent quite some time reading docs, issues, and sources I've come up with this workaround using `CurrentMessage` middleware:
```python3
@dramatiq.actor(queue_name="send_partner_webhook", max_retries=3)
def send_partner_webhook_actor(webhook: PartnerWebhook) -> None:
try:
send_partner_webhook(webhook)
except PartnerWebhookRequestFailedError as e:
message: dramatiq.Message = dramatiq.middleware.CurrentMessage.get_current_message()
retries_so_far: int = message.options.get("retries", 0)
backoff = {
0: 60_000, # backoff before first retry - 1 min
1: 300_000, # backoff before second retry - 5 min
2: 3_600_000, # backoff before third retry - 1 hour
}
if retries_so_far not in backoff:
raise e
raise dramatiq.errors.Retry(str(e), delay=backoff[retries_so_far])
```
I think it should be easier to achieve this behavior using an argument similar to `retry_when`:
```python3
def backoff_factory(retries_so_far: int) -> int:
backoff = {
0: 60_000, # backoff before first retry - 1 min
1: 300_000, # backoff before second retry - 5 min
2: 3_600_000, # backoff before third retry - 1 hour
}
return backoff.get(retries_so_far, 3_600_000)
@dramatiq.actor(queue_name="send_partner_webhook", max_retries=3, backoff_factory=backoff_factory)
def send_partner_webhook_actor(webhook: PartnerWebhook) -> None:
send_partner_webhook(webhook)
```
Thank you for your work, I hope that this feature request makes it into a future release!
Contributor guide
Assessment
This issue has not been assessed yet.