grpc / grpc/grpc-dotnet

Incomplete message error

Open
#1,778 5 comments 4 reactions 1 assignee Claimed by @JamesNK View on GitHub
bug investigate
Dominant language
C#
Stars
4.5k
Forks
836
Avg merge
6d 3h
Merged PRs (30d)
7

Description

### What version of gRPC and what language are you using?
grpc-dotnet client version 2.46.0 / C#

### What operating system (Linux, Windows,...) and version?
Linux

### What runtime / compiler are you using (e.g. .NET Core SDK version `dotnet --info`)
.NET 6 (6.0.202) SDK

### What did you do?
A client sends unary grpc requests that contain some bytes and expects back a response containing two int32 values.
From time to time the client receives an error for the request with grpc status code 13 and the message `Incomplete message` from the server.
To me it seems to be related to the server closing the HTTP2 connection by sending a GOAWAY.
Sadly i can't provide a reproducer as the issue only occurs seldomly.
I have attached some logs and stack traces that i have gathered when the error occurs

```
2022-05-30 18:57:23.878 <- Event Private.InternalDiagnostics.System.Net.Http - HandlerMessage : 56388208,0,0,SendWithVersionDetectionAndRetryAsync,Retry attempt 1 after connection failure. Connection exception: System.Net.Http.HttpRequestException: The request was aborted.
---> System.Net.Http.Http2ConnectionException: The HTTP/2 server sent invalid data on the connection. HTTP/2 error code 'NO_ERROR' (0x0).
--- End of inner exception stack trace ---
at System.Net.Http.Http2Connection.ThrowRetry(String message, Exception innerException)
at System.Net.Http.Http2Connection.Http2Stream.CheckResponseBodyState()
at System.Net.Http.Http2Connection.Http2Stream.TryEnsureHeaders()
at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)
at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) ->
```

```
2022-05-30 18:57:24.724 <- Event Private.InternalDiagnostics.System.Net.Http - HandlerMessage : 56388208,26131301,1,OnHeader,grpc-status: 13 ->
2022-05-30 18:57:24.725 <- Event Private.InternalDiagnostics.System.Net.Http - HandlerMessage : 56388208,26131301,1,OnHeader,grpc-message: Incomplete message. ->
```

### What did you expect to see?
A successful response for the grpc request

### What did you see instead?
An error response with grpc status code 13 and the message `Incomplete message`

I would also be thankful for guidance on how to further investigate the issue.

Kind regards,
Lars

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.