HarperFast / HarperFast/harper-pro

Use separate WS connections per originating-node subscription (avoid full audit scan resets on fail-over)

Open
#193 2 comments 0 reactions 1 assignee Claimed by @kriszyp View on GitHub
area:replication duplicate enhancement from-jira
Dominant language
JavaScript
Stars
3
Forks
0
Avg merge
1d 21h
Merged PRs (30d)
80

Description

**Architecture improvement toward per-node transaction logs.** When a node loses connectivity, Harper currently resets fail-over subscriptions to existing connections with a new array of originating node IDs and start times. This forces the receiving node to reset its range iterator to the *earliest* start time and scan the entire audit log from there with per-node filtering — potentially a very long scan, which has historically caused a multiplying problem of connection resets and additional fail-overs.

## Proposed approach

Use **separate WebSocket connections for each originating-node subscription**. If node X wants updates from nodes A, B, and C (all routed through node A), it opens 3 connections to A — one per source node — rather than 1 connection with a multi-node subscription array.

**Benefits:**
- Adding or updating subscription to node B/C doesn't interrupt the in-flight subscription to node A.
- Each connection can live on a different thread — better write parallelization.
- Each stream has its own start time and can be restarted independently.

**Drawbacks (noted in Jira):**
- Multiple audit iterators scanning the same store, each filtering for one node ID — likely less efficient than a single iterator filtering for multiple IDs.
- The proper fix is separate per-node transaction logs (CORE-2887 / [HarperFast/harper-pro#192](https://github.com/HarperFast/harper-pro/issues/192)), which would make this the natural architecture. This ticket is an intermediate step.

## Acceptance criteria

- Fail-over subscription resets for one source node do not reset or interrupt subscriptions for other source nodes.
- Long audit scans triggered by a single failed node don't cascade into additional fail-overs.
- Performance regression (if any from multiple iterators) is measured and within acceptable bounds.

## Relation to other work

- [CORE-2887](https://harperdb.atlassian.net/browse/CORE-2887) / [HarperFast/harper-pro#192](https://github.com/HarperFast/harper-pro/issues/192) — per-node transaction logs: the eventual architecture this aligns with
- [CORE-2641](https://harperdb.atlassian.net/browse/CORE-2641) — adaptive routing for better fan-out (requires multi-node subscription capability)

---

Jira fields: **Feature Type:** Tech Debt · **Business Impact:** Operational efficiency *(both suggested)*

🤖 Filed by Claude on behalf of Kris.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.