grpc / grpc/grpc-dotnet

Running server-streaming calls on HealthCheck API delay server shutdown

Open
#2,572 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
4.5k
Forks
836
Avg merge
6d 3h
Merged PRs (30d)
7

Description

### What version of gRPC and what language are you using?

Grpc 2.46.6
Grpc.AspNetCore.HealthChecks 2.66.0

### What operating system (Linux, Windows,...) and version?

Windows 11

### What runtime / compiler are you using (e.g. .NET Core SDK version `dotnet --info`)

.NET SDK 8.0.404

### What did you do?

We integrated the `Grpc.AspNetCore.HealthChecks` NuGet package and tested its gRPC API with clients using the `Watch` server-streaming RPC. To monitor the server's status, we implemented a simple health check class that verifies the server's running state. However, during server shutdown, we noticed a delay if clients were connected to the `Watch` RPC at the time.

### What did you expect to see?

A graceful shutdown that completes within a few seconds at most.

### What did you see instead?

When the server stops, the shutdown is significantly delayed if clients are connected to the `Watch` RPC at the time of shutdown. The still opened calls cause the server to reach its shutdown timeout. We retained the host shutdown timeout at 30 seconds.
The screenshot below shows a client connected to the `Watch` RPC. The shutdown was initiated at 12:07:26 (via the terminal) and completed at 12:07:56.
![Screenshot 2024-11-15 120818](https://github.com/user-attachments/assets/d82d75a3-0b78-4ae8-adc6-d9ba9e1f803f)

### Anything else we should know?

This issue is related to:

The problem is that running server-streaming calls are not informed about the shutdown. You will find my proposed solution later in the linked PR.

Contributor guide

Open the contributing guide

Research direction

Start with the Grpc.AspNetCore.HealthChecks Watch server-streaming RPC and the related issue #2357, then reproduce shutdown with a connected Watch client. Done means active Watch calls are informed of shutdown and the server stops within a few seconds instead of reaching the 30-second timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.