Can't connect to anything - System.Net.Sockets.SocketException

Start at WebSocket.setClientStream() and WebSocket.connect(), using the stack trace and TcpClient constructor as entry points. Compare this path with the working System.Net.WebSockets ClientWebSocket case described in the report, then reproduce against ws://localhost:4444/. Done means identifying and correcting the websocket-sharp connection failure without regressing normal connections.

I feel like there is a simple explanation and I'm doing something dumb here. But I'm just trying to setup a basic ws connection, my code looks like:

using (var ws = new WebSocket("ws://localhost:4444/"))
{
    ws.Log.Level = LogLevel.Trace;
    ws.OnMessage += (sender, e) => Console.WriteLine("ws says: " + e.Data);
    ws.Connect();
}

The output I get is:

3/31/2022 4:00:45 PM|Warn |Logger.set_Level|The current logging level has been changed to Trace.
3/31/2022 4:00:45 PM|Fatal|WebSocket.connect|The operation completed successfully.

3/31/2022 4:00:45 PM|Debug|WebSocket.connect|System.Net.Sockets.SocketException (0x80004005): The operation completed successfully.

                             at System.Net.Sockets.TcpClient..ctor (System.String hostname, System.Int32 port) [0x0006d] in <5307bcb359e140d4b454b70b3ff646a0>:0
                             at WebSocketSharp.WebSocket.setClientStream () [0x00066] in <c17dee5c18474d2096de0e4fb891b68d>:0
                             at WebSocketSharp.WebSocket.doHandshake () [0x00001] in <c17dee5c18474d2096de0e4fb891b68d>:0
                             at WebSocketSharp.WebSocket.connect () [0x000fa] in <c17dee5c18474d2096de0e4fb891b68d>:0
3/31/2022 4:00:45 PM|Trace|WebSocket.close|Begin closing the connection.
3/31/2022 4:00:45 PM|Debug|WebSocket.closeHandshake|Was clean?: False
                             sent: False
                             received: False
3/31/2022 4:00:45 PM|Trace|WebSocket.close|End closing the connection.
Disconnected to websocket server.

I can connect to it fine with the same address using basic System.Net.WebSockets library and the ClientWebSocket class. But websocket-sharp just gives that error and bails out no matter what I seem to change.