"Unmasked client to server frame" causes 502s

Hey @ptaoussanis — huge fan of http-kit. We use http-kit (both server and client) at my company, with Sente on top.

Wanted to figure out together why a particularly nasty issue is happening. Every time I see the following stack trace show up in our logs:

at java.base/java.lang.Thread.run(Thread.java:1583)
at org.httpkit.server.HttpServer.run(HttpServer.java:425)
at org.httpkit.server.HttpServer.doRead(HttpServer.java:307)
at org.httpkit.server.HttpServer.decodeWs(HttpServer.java:260)
at org.httpkit.server.WSDecoder.decode(WSDecoder.java:75)
org.httpkit.ProtocolException: unmasked client to server frame
Fri Mar 29 05:05:36 UTC 2024 [server-loop] ERROR - null

we get 502s in various subsequent requests to that server, even unrelated to WebSockets. The problem appears to linger for a few minutes, but it could be less. I believe there are still some successful requests that go through in that time frame — not every request to the affected server will 502 during that time, as far as I can tell, but I can double check this. (I haven't yet traced which requests 502 and which don't, and why.)

We run on AWS, and have our servers load balanced behind CloudFront. The 502 is being emitted from the AWS load balancer due to its failure to connect to a server that experienced the unmasked client to server frame issue.

Given that you have much more familiarity with http-kit's code than I, are you aware of some mechanism that might cause it to 502? It's almost as if it's defensively dropping requests for a time, or an event loop dies (I do see server-loop in the logs, after all) and needs a little time to pick itself back up.

Meanwhile I'll go dig into http-kit's code to see if I can gather any clues.

Also curious what you might recommend to mitigate this problem. Maybe a relevant exception needs to be caught somewhere? Presumably can't do much to mitigate the issue on the client end, as bad payloads happen sometimes.

Thanks! 🙏

Start in HttpServer.decodeWs and WSDecoder.decode, using the reported stack trace as the entry point. Trace how an unmasked client frame is handled and whether it can affect subsequent HTTP requests or the server loop. Done means identifying the cause of the later 502s and documenting or implementing a verified mitigation, with regression coverage if an existing test location is found.