grpc / grpc/grpc-dotnet

Error with `StatusCode` `Internal` thrown when a server is unavailable for a client running on .NET Framework 4.8 instead of `Unavailable`

Open
#2,581 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
4.5k
Forks
836
Avg merge
6d 3h
Merged PRs (30d)
7

Description

### What version of gRPC and what language are you using?
Grpc.Net.Client 2.67.0 and C#

### What operating system (Linux, Windows,...) and version?
Windows 11

### What runtime / compiler are you using (e.g. .NET Core SDK version `dotnet --info`)
.NET Framework 4.8

### What did you do?
I have a GRPC server written in Java Spring Boot. I want to ensure proper retries if the server is unavailable. From what I understand, the client will retry if a retryable status code is returned. I would have thought that if the server is unavailable, I should get a status code of `StatusCode.Unavailable` but instead I get `StatusCode.Internal`. Setting `StatusCode.Internal` as a retryable status code works so far as the retries are concerned but that error is thrown for a variety of reasons that I don't want to retry on. Following is how I am setting up the channel

```c#
WinHttpHandler winHttpHandler = new WinHttpHandler();
winHttpHandler.ServerCertificateValidationCallback = TlsValidationCallback;

GrpcChannel channel = GrpcChannel.ForAddress("https://localhost:9090", new GrpcChannelOptions
{
HttpHandler = winHttpHandler,
ServiceConfig = new ServiceConfig
{
MethodConfigs =
{
new MethodConfig
{
Names = { MethodName.Default }, RetryPolicy =
new RetryPolicy
{
InitialBackoff = TimeSpan.FromSeconds(2),
MaxBackoff = TimeSpan.FromSeconds(30),
BackoffMultiplier = 1.5,
RetryableStatusCodes = { StatusCode.Internal }
}
}
}
},
MaxRetryAttempts = 5
});

Client = new HelloService.HelloServiceClient(channel);

Client.SendMessahe(new HelloRequest { Name = "hello"});
```

### What did you expect to see?

I expected to get a `StatusCode.Unavailable` when the server is not running.

### What did you see instead?

I am getting a `StatusCode.Internal` instead.

The full error message is

```
Unhandled Exception: Grpc.Core.RpcException: Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. WinHttpException: Error 12029 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'A connection with the server could not be established'.", DebugException="System.Net.Http.HttpRequestException: An error occurred while sending the request.") ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12029 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'A connection with the server could not be established'.
at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
at System.Net.Http.WinHttpHandler.d__122.MoveNext()
--- End of inner exception stack trace ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at Grpc.Net.Client.Internal.GrpcCall`2.d__82.MoveNext() in /_/src/Grpc.Net.Client/Internal/GrpcCall.cs:line 508
--- End of inner exception stack trace ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Grpc.Net.Client.Internal.Retry.RetryCallBase`2.d__80.MoveNext() in /_/src/Grpc.Net.Client/Internal/Retry/RetryCallBase.cs:line 119
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Grpc.Net.Client.Internal.HttpClientCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request) in /_/src/Grpc.Net.Client/Internal/HttpClientCallInvoker.cs:line 153
at Grpc.Core.Interceptors.InterceptingCallInvoker.b__3_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx) in /_/src/Grpc.Core.Api/Interceptors/InterceptingCallInvoker.cs:line 53
at Grpc.Core.ClientBase.ClientBaseConfiguration.ClientBaseConfigurationInterceptor.BlockingUnaryCall[TRequest,TResponse](TRequest request, ClientInterceptorContext`2 context, BlockingUnaryCallContinuation`2 continuation) in /_/src/Grpc.Core.Api/ClientBase.cs:line 205
at Grpc.Core.Interceptors.InterceptingCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request) in /_/src/Grpc.Core.Api/Interceptors/InterceptingCallInvoker.cs:line 50
at MyRunner.RunnerService.RunnerServiceClient.ReplyRunner(HelloRequest request, CallOptions options) in my-folder\my-runner\MyRunner\obj\Debug\Protos\RunnerGrpc.cs:line 110
at MyRunner.RunnerService.RunnerServiceClient.ReplyRunner(HelloRequest request, Metadata headers, Nullable`1 deadline, CancellationToken cancellationToken) in my-folder\my-runner\MyRunner\obj\Debug\Protos\RunnerGrpc.cs:line 105
at MyRunner.RunnerCallbacks.ReplyRunner(RunnerReplyModel RunnerReplyModel) in my-folder\my-runner\MyRunner\RunnerCallbacks.cs:line 17
at Beezlabs.RPAHive.Lib.RPARunnerTemplate.RunRunner(RunnerExecutionModel RunnerExecutionModel)
at MyRunner.Program.d__0.MoveNext() in my-folder\my-runner\MyRunner\Program.cs:line 42
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at MyRunner.Program.(String[] args)
```

### Anything else we should know about your project / environment?
`TlsValidationCallback` is a function that I am using to validate a custom certificate which is why managing my own `WinHttpHandler` is neded.

Contributor guide

Open the contributing guide

Research direction

Reproduce the unavailable-server call on .NET Framework 4.8 with WinHttpHandler, then inspect Grpc.Net.Client/Internal/GrpcCall.cs around line 508 and Internal/Retry/RetryCallBase.cs around line 119. Done means a server that cannot be reached reports StatusCode.Unavailable, while unrelated Internal errors remain distinguishable for retry configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.