dotnet / dotnet/aspnetcore

Http/2 WebSocket streams do not close gracefully

Open
#42,328 2 comments 0 reactions 0 assignees View on GitHub
area-networking bug feature-websockets
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 10h
Merged PRs (30d)
281

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

Http/2 WebSocket streams do not close gracefully with END_STREAM flags, they send/receive RST_STREAM's instead.
```
the application completed without reading the entire request body.
sending RST_STREAM frame for stream ID 7 with length 4 and flags 0x0.
```

The client initiates the WebSocket close, the server responds, the server application unwinds and completes the response with and END_STREAM flag. At this point it sees that the client has not sent an END_STREAM flag so the server sends a RST_STREAM message to close the incoming channel.

The client seems to cache this failure and downgrade future WebSocket requests to HTTP/1.1. I've reproduced this with Chrome/Edge and Firefox. To go back to HTTP/2 often requires closing all browsers or switching to incognito.

(Note Firefox requires opting into H2 WebSockets in about:config)

If instead I have the server pause for 5s after sending the WS close message, the client sends a RST_STREAM message to abort the stream.

If instead I have kestrel skip sending the RST_STREAM message after the END_STREAM, the client never sends RST_STREAM or END_STREAM on that request, it seems to time out the connection soon afterwards.

### Expected Behavior

The client should be sending us an END_STREAM message immediately after sending the WS Close message. Why doesn't it?

https://datatracker.ietf.org/doc/html/rfc6455#section-7.1.1
To _Close the WebSocket Connection_, an endpoint closes the
underlying TCP connection. An endpoint SHOULD use a method that
cleanly closes the TCP connection, as well as the TLS session, if
applicable, discarding any trailing bytes that may have been
received. An endpoint MAY close the connection via any means
available when necessary, such as when under attack.

https://datatracker.ietf.org/doc/html/rfc8441#section-5
The HTTP/2 stream closure is also analogous to the TCP connection
closure of [[RFC6455](https://datatracker.ietf.org/doc/html/rfc6455)]. Orderly TCP-level closures are represented as
END_STREAM flags ([[RFC7540], Section 6.1](https://datatracker.ietf.org/doc/html/rfc7540#section-6.1)). RST exceptions are
represented with the RST_STREAM frame ([[RFC7540], Section 6.4](https://datatracker.ietf.org/doc/html/rfc7540#section-6.4)) with
the CANCEL error code ([[RFC7540], Section 7](https://datatracker.ietf.org/doc/html/rfc7540#section-7)).

### Steps To Reproduce

Repro app:
https://github.com/dotnet/aspnetcore/blob/146f49fdf09916d0c63e82570bfe059d7fb845e6/src/Middleware/WebSockets/samples/EchoApp/Program.cs

1. Connect the WebSocket using HTTP/2
2. Send a message
3. Close the WebSocket from the client
4. Review the server logs
5. Create a new WebSocket from the client. This is likely to downgrade to HTTP/1.1.

### Exceptions (if any)

_No response_

### .NET Version

7.0-preview6

### Anything else?

```
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[37]
Connection id "0HMIJKKPR7BUG" received HEADERS frame for stream ID 7 with length 154 and flags END_HEADERS, PRIORITY.
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/2 CONNECT https://localhost:5001/ - -
dbug: Microsoft.AspNetCore.WebSockets.WebSocketMiddleware[1]
WebSocket compression negotiation accepted with values 'permessage-deflate; client_max_window_bits=15'.
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[49]
Connection id "0HMIJKKPR7BUG" sending HEADERS frame for stream ID 7 with length 105 and flags END_HEADERS.
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[37]
Connection id "0HMIJKKPR7BUG" received DATA frame for stream ID 7 with length 7 and flags NONE.
dbug: EchoApp[0]
Received Frame Text: Len=0, Fin=True:
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[49]
Connection id "0HMIJKKPR7BUG" sending DATA frame for stream ID 7 with length 3 and flags NONE.
dbug: EchoApp[0]
Sent Frame Text: Len=0, Fin=True:
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[37]
Connection id "0HMIJKKPR7BUG" received WINDOW_UPDATE frame for stream ID 0 with length 4 and flags 0x0.
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[37]
Connection id "0HMIJKKPR7BUG" received WINDOW_UPDATE frame for stream ID 7 with length 4 and flags 0x0.
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[37]
Connection id "0HMIJKKPR7BUG" received DATA frame for stream ID 7 with length 27 and flags NONE.
dbug: EchoApp[0]
Received Frame Close: NormalClosure Closing from client
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[49]
Connection id "0HMIJKKPR7BUG" sending DATA frame for stream ID 7 with length 23 and flags NONE.
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished HTTP/2 CONNECT https://localhost:5001/ - - - 200 - - 6890.5193ms
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[49]
Connection id "0HMIJKKPR7BUG" sending DATA frame for stream ID 7 with length 0 and flags END_STREAM.
info: Microsoft.AspNetCore.Server.Kestrel[32]
Connection id "0HMIJKKPR7BUG", Request id "0HMIJKKPR7BUG:00000007": the application completed without reading the entire request body.
trce: Microsoft.AspNetCore.Server.Kestrel.Http2[49]
Connection id "0HMIJKKPR7BUG" sending RST_STREAM frame for stream ID 7 with length 4 and flags 0x0.
```

Contributor guide

Open the contributing guide

Research direction

Start with the repro in src/Middleware/WebSockets/samples/EchoApp/Program.cs and reproduce the HTTP/2 close sequence while reviewing the Kestrel HTTP/2 logs. Compare the observed END_STREAM and RST_STREAM behavior with RFC 8441 section 5. Done means an orderly WebSocket close no longer produces the reported RST_STREAM or causes later requests to downgrade unexpectedly.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend-api-design, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.