dotnet / dotnet/extensions

gRPC client with AddStandardResilienceHandler reports Unavailable instead of Cancelled when the caller cancels during a transient failure

Open
#7,741 0 comments 0 reactions 0 assignees View on GitHub
bug untriaged
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

### Description

A gRPC client configured with `AddGrpcClient(...).AddStandardResilienceHandler()` and cancelled by the caller while an attempt is in flight fails with `StatusCode.Unavailable` if that attempt completes with a transient HTTP failure. It should be `StatusCode.Cancelled`.

Any failure the retry strategy would have retried triggers it (with the default `HttpRetryStrategyOptions` that's 5xx, 408, 429 and `HttpRequestException`). The reported status varies with the failure, but it's never `Cancelled`.

The cause is in the Polly version pinned in `eng/packages/General.props` (8.4.2). Its `RetryResilienceStrategy` returns the last, failed outcome instead of throwing when the token is cancelled:

```csharp
if (context.CancellationToken.IsCancellationRequested || isLastAttempt || !handle)
{
return outcome;
}
```

Polly changed this in 8.5.2 (App-vNext/Polly#2456): cancellation now surfaces as `OperationCanceledException`.

### Reproduction Steps

The simplest console app: https://github.com/gitmln/DotnetExtensionsRepro

```
dotnet run # Polly 8.4.2, as pinned by Microsoft.Extensions.Http.Resilience 9.10.0
dotnet run -p:PollyVersion=8.7.0
```

The relevant part:

```csharp
services
.AddGrpcClient(options => options.Address = new Uri("https://dummy"))
.ConfigurePrimaryHttpMessageHandler(() => new CancelThenFailHandler(cts))
.AddStandardResilienceHandler();

// CancelThenFailHandler: cancel the caller's token, then complete the attempt with a 503
protected override Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
cts.Cancel();
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.ServiceUnavailable) { Version = HttpVersion.Version20 });
}

await client.SayHelloAsync("dummy", cts.Token); // RpcException — which StatusCode?
```

The call fails with Cancelled only on Polly 8.7.0.

The same scenario as a test in this repo, using the generated `Greeter.GreeterClient` and the existing `GrpcResilienceTests` fixture: gitmln/dotnet-extensions@ab484b90f973e073f091eb3d35be4fc657b94a02. On `main` (0fe52dfa38) both the sync and async variants fail; with the Polly packages at 8.7.0 (gitmln/dotnet-extensions@754ada5d3f62f17250aae435d9b174f1cdfac2bf) they pass.

### Expected behavior

`RpcException` with `StatusCode.Cancelled`.

### Actual behavior

`RpcException` with `StatusCode.Unavailable`.

### Regression?

No. Present for as long as a Polly older than 8.5.2 has been referenced.

### Known Workarounds

None that I can see.

### Configuration

- `main` at 0fe52dfa38, Polly 8.4.2 via `eng/packages/General.props`
- `net8.0` on Microsoft.NETCore.App / AspNetCore.App 8.0.30; SDK 10.0.400 (`global.json` `rollForward` relaxed locally), Windows 11 10.0.26200, win-x64

### Other information

I've seen #7719. This isn't an argument about which Polly to ship long term — only that the pinned one has a user-visible bug in a documented scenario. Branch with the test and the bump, ready for a PR if that's useful: https://github.com/gitmln/dotnet-extensions/tree/polly-retry-cancellation-issue

_I used AI/Claude Code for the investigation and to draft the test_

Contributor guide

Open the contributing guide

Research direction

Start with the Polly version pinned in eng/packages/General.props, then inspect the existing GrpcResilienceTests fixture and the generated Greeter.GreeterClient scenario described in the issue. Run the sync and async cancellation tests with the current dependency; done means both report RpcException with StatusCode.Cancelled after the supported Polly update.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, grpc
Domain
api, backend, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.