ROCm / ROCm/FastFlowLM

Server stops accepting connections after 10 client aborts — connection slot leak ("Connection limit reached (10)")

Open
#680 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

Summary

flm serve permanently rejects all new connections after roughly 10 client-side aborts, even though no TCP connections remain open. Only a process restart recovers the server.

Environment
  • FastFlowLM 1.0.1
  • Ubuntu 26.04, kernel 7.0.0-29, Ryzen AI MAX+ 395 (Strix Halo, XDNA2)
  • Ryzen AI XRT 2.25.37, amdxdna DKMS 2.25.260102.56, NPU firmware 1.1.2.65
  • Serving: flm serve gemma4-it:e2b --host 0.0.0.0 --port 52625 (systemd user service)
Steps to reproduce
  1. Start flm serve <model>.
  2. From an HTTP client, send a /v1/chat/completions request with a long generation, then abort the connection mid-stream (client timeout / connection close) — in our case the client is an LLM-harness advisor with a request timeout shorter than cold generation (~20 s).
  3. Repeat ~10 times over the service lifetime.
  4. All subsequent connections are rejected. ss shows zero established sockets on the port, yet the server logs Connection limit reached (10), rejecting new connection for every attempt.
Logs
[🔵 ]  NPU Lock Released!
[🔒 ]  TCP connection closed - Remote endpoint unavailable
[LOG]  Connection limit reached (10), rejecting new connection   ← repeats forever
Expected behavior

Aborted connections release their slot; the server keeps accepting new connections indefinitely.

Actual behavior

The internal connection counter is never decremented on client abort. After 10 aborts the server is deaf until restart.

Workaround

systemd watchdog probing /v1/models every 5 min, restarting the service on failure.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the flm serve request and connection-handling entry points, then reproduce the issue with repeated aborted /v1/chat/completions streams. Trace the connection counter through the TCP connection close path and verify that client aborts release slots; done means more than 10 aborts still allow new connections without a restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.