HarperFast / HarperFast/harper

SSE subscription teardown is deferred until the next event → idle subscriber leaks listener + socket

Open
#1,399 0 comments 0 reactions 0 assignees View on GitHub
area:rest-api bug
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

When an SSE / `connect`-based live subscriber disconnects, the subscription's teardown (`'close'` → remove the hub listener, release the socket) does **not** fire on the disconnect itself — it fires only when the **next event is delivered** to that subscription. So a subscriber that disconnects and then receives no further events leaves its hub listener + socket handle registered **indefinitely**.

## Severity

Medium — bounded under steady traffic (the residual drains as events flow; no unbounded growth under load), but an idle/abandoned subscriber on a low-traffic topic pins one listener + one socket handle until the next event that never comes. At scale (many short-lived or abandoned SSE clients on quiet topics) this accumulates. This is the concrete mechanism behind the previously-theorized "unbounded live-tail queue per stalled subscriber."

## Repro

1. Open an SSE subscription to a Table topic (`Accept: text/event-stream`).
2. Receive one event.
3. Disconnect the client (drop the socket) — **do not** send further events to that subscription.
4. Observe (per-worker) that the hub listener count / open-handle count did **not** decrement; it only drops once a subsequent event flushes the parked iterator.

(Controlled curl repro: open SSE, deliver one event, disconnect → `closed=0`, listener/handle counters incremented and not reclaimed.)

Contrast: async-generator streaming responses tear down promptly on mid-stream disconnect (their `finally{}` runs because they self-resolve via their own timers) — the asymmetry pinpoints the cause.

## Root cause

The SSE response body is `Readable.from(transformIterable(subscription))`, destroyed on client close via `nodeResponse.on('close', () => body.destroy())` (`server/http.ts:464-467`). But the subscription iterator's `next()` parks on a promise that only resolves on `send()` (`resources/IterableEventQueue.ts`); `Readable.from`'s destroy cannot invoke `iterator.return()` (→ `'close'`) until that parked `next()` settles. So teardown waits for the next event.

## Recommendation

Tear down on client disconnect without waiting for the next event — adopt the WebSocket path's explicit `iterator.return()`-on-disconnect wiring (`server/REST.ts:338-342`) for the SSE/`Readable` path, or make `body.destroy()` actively settle/cancel the parked `next()`.

---
*Surfaced by the QA-explorer campaign against Harper `7aaa5a152`. The cited code paths (`server/http.ts`, `resources/IterableEventQueue.ts`) are unchanged vs `main` @`6797f091d`. Companion note (D-065): a Table SSE subscription also withholds HTTP response headers until its initial query yields ≥1 row, so subscribing to an empty table blocks at connect. Filed by Claude (Opus 4.8) for @kris.*

Contributor guide

Open the contributing guide

Research direction

Start with the SSE disconnect handling in server/http.ts:464-467, then compare the explicit iterator.return() wiring in server/REST.ts:338-342 and the parked next() behavior in resources/IterableEventQueue.ts. Use the controlled curl reproduction to verify that disconnecting without another event releases the hub listener and socket handle immediately.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
api, backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.