Minimum timeout for retry policy
- Dominant language
- C#
- Stars
- 4.5k
- Forks
- 836
- Avg merge
- 6d 3h
- Merged PRs (30d)
- 7
Description
### Is your feature request related to a problem? Please describe.
Our test environment fails to get going, we think it's because sometimes all the retries happen in quick succession. We have various scripts that orchestrate multiple server components. There are dependencies between these components and sometimes it takes a few seconds for them to all come online, sometimes longer because reasons.
### Describe the solution you'd like
I'd like to extend the retry policy with a minimum delay between retries.
### Describe alternatives you've considered
Write my own gRPC stack...
I looked into the [retry code](https://github.com/grpc/grpc-dotnet/blob/492158d5cbd2c83d8d546849f88701f368058d5e/src/Grpc.Net.Client/Internal/Retry/RetryCall.cs#L209-L217), and there's nothing I can do from my end.
### Additional context
It might have been a mistake to not provide a minimum delay between retries because now there's no way to ensure that at least X seconds is spent waiting for services to come online. The way this is described [elsewhere](https://github.com/grpc/proposal/blob/master/A6-client-retries.md#exponential-backoff) makes me think this might have been an oversight?
I ran some experiments with the default retry policy documented [here](https://learn.microsoft.com/en-us/aspnet/core/grpc/retries?view=aspnetcore-9.0#configure-a-grpc-retry-policy).
Basically, there's about 1/100 chance that all retries happen within 1 second (because there's no minimum).
I think there needs to be a minimum, it can default to zero, but at least this way we can ensure that there's a minimum period where it's waiting before giving up.
What we believe is happening in our test environment is that sometimes it takes a bit longer for various server components to go online which leads to intermittent startup failures which is causing CI/CD to fail more or less randomly.
Contributor guide
Research direction
Start with src/Grpc.Net.Client/Internal/Retry/RetryCall.cs around lines 209-217, then compare its behavior with the A6 client-retries exponential-backoff description. Trace how the documented default retry policy is configured and identify the policy surface and tests that would need to cover a configurable minimum delay, including the default-zero behavior. Done means the requested delay is supported and its retry timing is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, grpc
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100