dotnet / dotnet/extensions

HttpClientResiliencePredicates.IsHttpConnectionTimeout will always false after aot publish

Open
#7,188 0 comments 0 reactions 0 assignees View on GitHub
area-resilience bug untriaged
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

### Description

In the latest version of `Microsoft.Extensions.Http.Resilience`, `HttpClientResiliencePredicates.IsHttpConnectionTimeout` assume that a _**`OperationCanceledException` which `Source` is `System.Private.CoreLib`**_ and `InnerException` is `TimeoutException` is a connection timeout exception.

https://github.com/dotnet/extensions/blob/9e8d935cf0e7014d338ebdb7d6d7d4d348d62021/src/Libraries/Microsoft.Extensions.Http.Resilience/Polly/HttpClientResiliencePredicates.cs#L53-L55

It usually works well, but _**after aot publish. The `Exception.Source` will always return ``**_ (If metadata for current method cannot be established).

https://github.com/dotnet/runtime/blob/138d5fe8f32e954a3a834be6360a834688130d4f/src/libraries/System.Private.CoreLib/src/System/Exception.cs#L94-L100
```csharp
public virtual string? Source
{
[UnconditionalSuppressMessage("ReflectionAnalysis", "IL2026:RequiresUnreferencedCode",
Justification = "The API will return if the metadata for current method cannot be established.")]
get => _source ??= HasBeenThrown ? (TargetSite?.Module.Assembly.GetName().Name ?? "") : null;
set => _source = value;
}
```

Which case any resilience pipeline using `HttpRetryStrategyOptions` with default `ShouldHandle` will never retry if connection timeout error occured after aot publish.

### Reproduction Steps

Using follow program:

```csharp
#!/usr/local/share/dotnet/dotnet run

#:package Microsoft.Extensions.Http.Resilience@10.1.0
#:package Microsoft.Extensions.Logging.Console@10.0.0

using Microsoft.Extensions.Http.Resilience;
using Microsoft.Extensions.Logging;
using Polly;

using ILoggerFactory factory = LoggerFactory.Create(builder => builder.AddConsole());
ILogger logger = factory.CreateLogger("Program");

var socketHttpHandler = new SocketsHttpHandler
{
ConnectTimeout = TimeSpan.FromMilliseconds(1) // to trigger connect timeout
};
var retryPipeline = new ResiliencePipelineBuilder()
.AddRetry(new HttpRetryStrategyOptions
{
MaxRetryAttempts = 1,
// just fail fast
Delay = TimeSpan.FromMilliseconds(1),
BackoffType = DelayBackoffType.Linear
})
.ConfigureTelemetry(factory)
.Build();

var client = new HttpClient(new ResilienceHandler(retryPipeline)
{
InnerHandler = socketHttpHandler
});

try
{
await client.GetAsync("https://example.com");
}
catch (Exception ex)
{
logger.LogInformation("Exception.Source: {ExceptionSource}", ex.Source); // Should be `System.Private.CoreLib`, but `` after aot publish
}
```

**Just `dotnet run`**

```plaintext
dotnet run .\retry-fail-aot.cs
warn: Polly[3]
Execution attempt. Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.', Handled: 'True', Attempt: '0', Execution Time: 30.3317ms
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException oce)
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<b__3_0>d.MoveNext()
warn: Polly[0]
Resilience event occurred. EventName: 'OnRetry', Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.'
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException oce)
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<b__3_0>d.MoveNext()
fail: Polly[3]
Execution attempt. Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.', Handled: 'True', Attempt: '1', Execution Time: 17.4763ms
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException oce)
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<b__3_0>d.MoveNext()
info: Program[0]
Exception.Source: System.Private.CoreLib
```

**After aot publish**
```plaintext
.\retry-fail-aot.exe
info: Polly[3]
Execution attempt. Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.', Handled: 'False', Attempt: '0', Execution Time: 21.8876ms
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException) + 0xb3
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<b__3_0>d.MoveNext() + 0x91
info: Program[0]
Exception.Source:
```

`Resilience event occurred. EventName: 'OnRetry'` was missing from program after aot publish. And `Exception.Source` is ``.

### Expected behavior

`HttpClientResiliencePredicates.IsHttpConnectionTimeout` should return true if exception is connect timeout exception after aot publish.

### Actual behavior

`HttpClientResiliencePredicates.IsHttpConnectionTimeout` always return false after aot publish.

### Regression?

`10.0.0` have same issue.

### Known Workarounds

Write a own `HttpClientResiliencePredicates` and replace `IsHttpConnectionTimeout()` to following version:

```csharp
internal static bool IsHttpConnectionTimeout(in Outcome outcome, in CancellationToken cancellationToken)
=> !cancellationToken.IsCancellationRequested
&& outcome.Exception is OperationCanceledException { InnerException: TimeoutException };
```

```csharp
new HttpRetryStrategyOptions()
{
ShouldHandle = args => new ValueTask(CustomHttpClientResiliencePredicates.IsTransient(args.Outcome, args.Context.CancellationToken))
}
```

### Configuration

```
.NET SDK:
Version: 10.0.101
Commit: fad253f51b
Workload version: 10.0.100-manifests.c57ac48b
MSBuild version: 18.0.6+fad253f51

OS Name: Windows
OS Version: 10.0.26100
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\10.0.101\

No .net workloads was installed.

Host:
Version: 10.0.1
Architecture: x64
Commit: fad253f51b

.NET SDKs installed:
9.0.305 [C:\Program Files\dotnet\sdk]
10.0.101 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 10.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.22 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 10.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 8.0.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 10.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found
```

### Other information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.