Infinite IO block present in ReadBytesAsync causes slow or malicious connections to block entire server
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- csharp
- Domain
- backend, networking
Research direction
Start in Ext.cs at ReadBytesAsync and trace how WebSocket.startReceiving() reaches receive() and the stream.EndRead(ar) call. Reproduce the behavior with a deliberately slow connection and determine whether the blocking read prevents other connections from reaching OnMessage. Done should prevent one slow or malicious connection from blocking the server while preserving appropriate connection handling.
Written by the indexing model from the issue text.
Description
In "ReadBytesAsync" in "Ext.cs", there is a line like:
var nread = stream.EndRead(ar);
This is an infinite blocking IO, and if a user has abysmally slow internet or is maliciously slowing down their connection while still technically "connected", this can lock down the whole server. This was tested in the debugger: if one connection is blocking on the above line, no other connections will call the OnMessage event in the WebSocket function startReceiving(). They all get stuck on the receive (); function call, and never finish reading their frame due to the block. I'm not sure if it's the stream itself blocking everyone else or if this block is within code that's holding onto a global lock, but all connections have to wait when one connection is stuck here.
I'm running a chat server and a user in another country is able to block the entire server simply by watching an HD video. His connection starts dropping packets like crazy, and if he sends a message during this time, the server waits until he sends the full message, which could take a very long time. The server would basically get entirely locked down until he reset his connection, and this happened about 10 times a day. After logging every single lock on my end and extensive debugging, I'm reasonably certain the above blocking IO is the cause (but I could be wrong!).
I think the fix is easy though! You can set a blocking timeout on the WebSocket class's _stream like so:
//Adjust these to fit the need, or simply make it a parameter in the server constructor
_stream.ReadTimeout = 2000; //These are milliseconds
_stream.WriteTimeout = 2000;
Thus, if the stream.EndRead(ar); call blocks the stream for more than 2 seconds, the connection is closed. I haven't had any server blocks since changing this, and only individual users get disconnected if the server isn't able to read their message in time (it's really just the one user with horrible internet). For my needs this is appropriate, as I'd rather have individual users get disconnected than to lockup the whole server. However, this may not be the proper solution for everyone. Maybe if it were made an option when constructing the server, it could solve problems like this when they arise. I don't know, it's up to you!
- Dominant language
- C#
- Stars
- 6.1k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from sta/websocket-sharp
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
sta/websocket-sharp#201 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 45/100
sta/websocket-sharp#762 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
sta/websocket-sharp#761 ·
-
Bad handling of sending stream makes lib not RFC compatible and KeepClean useless in some cases. Open
Difficulty 4/5 3-5 days Newbie friendliness 42/100
sta/websocket-sharp#760 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
sta/websocket-sharp#759 ·
All issues in sta/websocket-sharp
Similar issues
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 75/100
sillsdev/languageforge-lexbox#2665 ·
-
bug documentation frontend
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
azurenoops/spin_agent#975 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
:watch: Not Triaged 11.0 fundamentals/subsvc
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
dotnet/AspNetCore.Docs#37699 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
SubtitleEdit/subtitleedit#15108 · 1 comment ·