HarperFast / HarperFast/harper-pro

Replication subscription can remain ping-alive while sender setup never completes

Open
#642 0 comments 0 reactions 1 assignee Claimed by @kriszyp View on GitHub
Dominant language
JavaScript
Stars
3
Forks
0
Avg merge
1d 21h
Merged PRs (30d)
80

Description

## Problem

In a two-node v5.2.0-beta.4 cluster, one replication direction for one user database remained WebSocket-connected with fresh ping/pong liveness for more than two days while making no application progress. The receiver reported a zero received version, no received remote time, and an old durable cursor. A controlled write did not arrive or move the cursor; the reverse direction and other databases continued replicating.

The sender's `SUBSCRIPTION_REQUEST` handler gates `DB_SCHEMA` and the entire replay loop behind two unbounded asynchronous setup operations: the dynamic `hdb_nodes` authorization subscription and the database subscription placeholder. Rejections reach the existing catch, but a promise that never settles leaves the socket healthy at the transport layer while sending no replication frames. This is invisible to the connection-truth machinery tracked in #431 because that machinery correctly sees a live socket; this is an application-progress failure.

## Proposed fix

- Bound both sender-side setup waits and close/retry from the receiver's last durable cursor when either times out.
- Add a worker-local, one-shot subscription-setup watchdog. Arm it for a non-empty outbound `SUBSCRIPTION_REQUEST`; satisfy it on `DB_SCHEMA` for that database or any real subscription-progress frame; do not satisfy it on ping/pong or `NODE_NAME`.
- Do not treat a zero receive timestamp as stalled and do not advance a durable cursor merely to acknowledge setup.
- Log which sender setup wait timed out and include existing connection-truth telemetry when the receiver watchdog fires.

## Acceptance criteria

- A deliberately never-settling sender setup promise cannot leave a connected subscription permanently idle; the link reconnects and resumes from the durable cursor.
- A caught-up, zero-traffic subscription does not reconnect-churn.
- A large initial replay is not timed out after its schema/setup acknowledgement.
- Reverse-direction and sibling-database subscriptions are unaffected.
- Unit coverage pins setup-signal classification, timeout/cancel/rearm behavior, and sender wait bounds.
- An integration regression reproduces the ping-alive/no-application-progress failure on the base revision and proves recovery with the fix.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.