socketio / socketio/socket.io

Connection State Recovery fails for long-lived connections

Open
#5,282 21 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

to triage
Dominant language
TypeScript
Stars
63.2k
Forks
10.3k
Avg merge
11d 20h
Merged PRs (30d)
2

Description

Describe the bug
If the server does not periodically send the client messages, connection state recovery may fail unexpectedly.

To Reproduce

Socket.IO server version: 4.7.5, in-memory adapter

Server

import { Server } from "socket.io";

const io = new Server(3000, {
  connectionStateRecovery: {
    // the backup duration of the sessions and the packets
    maxDisconnectionDuration: 2 * 60 * 1000,
    // whether to skip middlewares upon successful recovery
    skipMiddlewares: true,
  }
});

io.on("connection", (socket) => {
  console.log(`connect ${socket.id}`);
  socket.emit("hello client", "Assigning offset to client");

  socket.on("disconnect", () => {
    console.log(`disconnect ${socket.id}`);
  });
});

We noticed this using a bespoke client, but appears to be a server-side issue. I've tried to mock up the client-side code accordingly.
Client

import { io } from "socket.io-client";

const socket = io({
  reconnectionDelay: 10000, // defaults to 1000
  reconnectionDelayMax: 10000 // defaults to 5000
});

socket.on("connect", () => {
  console.log("recovered?", socket.recovered);

  setTimeout(() => {
    if (socket.io.engine) {
      // close the low-level connection and trigger a reconnection
      socket.io.engine.close();
    }
  }, 3 * 60 * 1000);
});

Steps:

  1. Startup server, connect client to server, server emits an event to set client's recovery offset.
  2. Wait for 2x maxDisconnectionDuration without sending any events.
  3. Eventually Server will purge all buffered events on its side within maxDisconnectionDuration (+/- a minute or two).
  4. Cause client to reconnect
  5. Server cannot find appropriate offset because it has purged all buffered events.
  6. Client is assigned new ID.

Expected behavior
Connection State Recovery should succeed within maxDisconnectionDuration of disconnection, as long as the server must send at least one event, in order to initialize the offset on the client side.

It might help if server were to store offset of last queued packet in a separate variable, which could be checked even if restoreSession() cannot find the offset in the queue.

If server no longer relies upon pending message queue, we could then use Engine.IO PING/PONG events to trim queue(Once PONG is received, all events sent before the PING that triggered PONG are known to have arrived at the client).

Platform:

  • Client Device: electronic board game
  • OS: FreeRTOS

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the server-side connection state recovery flow, especially restoreSession(), and inspect how the buffered packet queue is purged after maxDisconnectionDuration. Reproduce the long-idle scenario from the issue, then verify recovery still succeeds when no events are sent during the disconnection window.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, typescript
Domain
backend, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.