Infinite IO block present in ReadBytesAsync causes slow or malicious connections to block entire server

Open
#167 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
csharp

Research direction

Start in Ext.cs at ReadBytesAsync and trace how WebSocket.startReceiving() reaches receive() and the stream.EndRead(ar) call. Reproduce the behavior with a deliberately slow connection and determine whether the blocking read prevents other connections from reaching OnMessage. Done should prevent one slow or malicious connection from blocking the server while preserving appropriate connection handling.

Written by the indexing model from the issue text.

Description

In "ReadBytesAsync" in "Ext.cs", there is a line like:

var nread = stream.EndRead(ar);

This is an infinite blocking IO, and if a user has abysmally slow internet or is maliciously slowing down their connection while still technically "connected", this can lock down the whole server. This was tested in the debugger: if one connection is blocking on the above line, no other connections will call the OnMessage event in the WebSocket function startReceiving(). They all get stuck on the receive (); function call, and never finish reading their frame due to the block. I'm not sure if it's the stream itself blocking everyone else or if this block is within code that's holding onto a global lock, but all connections have to wait when one connection is stuck here.

I'm running a chat server and a user in another country is able to block the entire server simply by watching an HD video. His connection starts dropping packets like crazy, and if he sends a message during this time, the server waits until he sends the full message, which could take a very long time. The server would basically get entirely locked down until he reset his connection, and this happened about 10 times a day. After logging every single lock on my end and extensive debugging, I'm reasonably certain the above blocking IO is the cause (but I could be wrong!).

I think the fix is easy though! You can set a blocking timeout on the WebSocket class's _stream like so:

//Adjust these to fit the need, or simply make it a parameter in the server constructor
_stream.ReadTimeout = 2000;   //These are milliseconds
_stream.WriteTimeout = 2000;

Thus, if the stream.EndRead(ar); call blocks the stream for more than 2 seconds, the connection is closed. I haven't had any server blocks since changing this, and only individual users get disconnected if the server isn't able to read their message in time (it's really just the one user with horrible internet). For my needs this is appropriate, as I'd rather have individual users get disconnected than to lockup the whole server. However, this may not be the proper solution for everyone. Maybe if it were made an option when constructing the server, it could solve problems like this when they arise. I don't know, it's up to you!

Dominant language
C#
Stars
6.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from sta/websocket-sharp

All issues in sta/websocket-sharp

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.