HarperFast / HarperFast/harper-pro
Replication subscription can remain ping-alive while sender setup never completes
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 0
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 80
Description
## Problem
In a two-node v5.2.0-beta.4 cluster, one replication direction for one user database remained WebSocket-connected with fresh ping/pong liveness for more than two days while making no application progress. The receiver reported a zero received version, no received remote time, and an old durable cursor. A controlled write did not arrive or move the cursor; the reverse direction and other databases continued replicating.
The sender's `SUBSCRIPTION_REQUEST` handler gates `DB_SCHEMA` and the entire replay loop behind two unbounded asynchronous setup operations: the dynamic `hdb_nodes` authorization subscription and the database subscription placeholder. Rejections reach the existing catch, but a promise that never settles leaves the socket healthy at the transport layer while sending no replication frames. This is invisible to the connection-truth machinery tracked in #431 because that machinery correctly sees a live socket; this is an application-progress failure.
## Proposed fix
- Bound both sender-side setup waits and close/retry from the receiver's last durable cursor when either times out.
- Add a worker-local, one-shot subscription-setup watchdog. Arm it for a non-empty outbound `SUBSCRIPTION_REQUEST`; satisfy it on `DB_SCHEMA` for that database or any real subscription-progress frame; do not satisfy it on ping/pong or `NODE_NAME`.
- Do not treat a zero receive timestamp as stalled and do not advance a durable cursor merely to acknowledge setup.
- Log which sender setup wait timed out and include existing connection-truth telemetry when the receiver watchdog fires.
## Acceptance criteria
- A deliberately never-settling sender setup promise cannot leave a connected subscription permanently idle; the link reconnects and resumes from the durable cursor.
- A caught-up, zero-traffic subscription does not reconnect-churn.
- A large initial replay is not timed out after its schema/setup acknowledgement.
- Reverse-direction and sibling-database subscriptions are unaffected.
- Unit coverage pins setup-signal classification, timeout/cancel/rearm behavior, and sender wait bounds.
- An integration regression reproduces the ping-alive/no-application-progress failure on the base revision and proves recovery with the fix.
Contributor guide
Assessment
This issue has not been assessed yet.