HarperFast / HarperFast/harper-pro

feat(replication): bridge enablement — add_node with full-copy and table exclusion

Open
#239 0 comments 0 reactions 1 assignee Claimed by @kriszyp View on GitHub
Dominant language
JavaScript
Stars
3
Forks
0
Avg merge
1d 21h
Merged PRs (30d)
80

Description

## Motivation

The v4 → v5 Fabric migration needs to bridge an established v5 cluster to a v4 source. Two related capabilities are needed on the v5 side:

1. **Replicate the `system` database across the bridge** (so `hdb_user` / `hdb_role` / `hdb_session` carry over) but **exclude `system.hdb_nodes`**, which represents the source cluster's topology and is toxic on the target.
2. **Trigger full-table-copy from v4 → v5** even when the v5 bridge node is already part of an established v5 mesh. Today, full-table-copy is gated on `node.isLeader === true`, which is only set by the `cloneNode` bootstrap workflow — and `cloneNode` requires a fresh install (no existing peers, no signed cert). That's not the bridge's situation.

This issue tracks both as one body of work because they ship together for the migration use case (PR [#240](https://github.com/HarperFast/harper-pro/pull/240) covers part of #1; the rest is below).

## Why not cloneNode-from-v4 for the bridge

The original runbook §3a put cloneNode-against-v4 on the bridge. That's wrong: by §1 the bridge is a meshed v5 cluster member with its own cert, peers, and `hdb_nodes` row. cloneNode wasn't designed to re-bootstrap an already-meshed node, and forcing it to work invites bugs that only appear at customer cutover. The right primitive for "connect an established node to a peer cluster" is `add_node`, with the additional capability to request full-table-copy and table exclusion.

(PR #237's `cloneSchemas` and the `cloneFromLegacy` test in PR #235 remain valid for the fresh-install cloneNode path, but they are no longer load-bearing for the migration.)

## Proposed API

### Part A — `excludeTables` on routes (PR #240, in progress)

Static-route config (already implemented in PR #240):

```yaml
replication:
routes:
- hostname: v4-bridge.example.com
port: 9933
sendsTo:
- target: v4-bridge
database: system
excludeTables: [hdb_nodes]
receivesFrom:
- source: v4-bridge
database: system
excludeTables: [hdb_nodes]
```

PR #240 implements three protection layers (subscription filter, sender-side skip, receiver drop). **Gap to close before merge:** the sender-side skip currently has no effect on dynamically-added nodes because dynamic routes don't flow through `routeByHostname`. The PR author marked this as "acceptable for the migration use case" — that assumption inverts under Part B below: `add_node` *is* the migration use case, so the sender-side skip needs to work for dynamic routes too.

### Part B — `add_node` accepts `isLeader` and `excludeTables`

Extend the `add_node` operation:

```json
{
"operation": "add_node",
"hostname": "v4-source.example.com",
"port": 9933,
"isLeader": true,
"excludeTables": [{ "database": "system", "table": "hdb_nodes" }],
"authorization": { "username": "...", "password": "..." }
}
```

- `isLeader: true` sets `isLeader: true` on the v5 side's record of the v4 peer. On connect, the v4 sender then takes the full-table-copy path (already wired at `replicationConnection.ts:1463`) and walks every table's primary store — covering data older than v4's audit-log retention. Asymmetric on purpose: v5 → v4 doesn't need full-copy because v5 only holds what v4 already shipped.
- `excludeTables` on `add_node` lands in the same dynamic-node config that the sender-side skip needs to read (closing the Part A gap by construction — if we thread `excludeTables` through `add_node` correctly, the dynamic-route gap goes away).

## Implementation pointers

- **Sender filtering hook**: `shouldReplicateFromNode(node, databaseName)` in [replication/knownNodes.ts:132](replication/knownNodes.ts#L132) needs a table parameter. Callers in `replicator.ts:569` and `subscriptionManager.ts:238` walk per-database; thread the table down to the subscription-open path.
- **Receiver-side enforcement**: sender-side filtering is necessary but not sufficient — a peer running an older build (e.g. v4) won't honor `excludeTables`, so the receiver must still drop incoming writes for excluded tables. PR #240 has this; preserve it.
- **`add_node` `isLeader` parameter**: today `isLeader: true` is set only by the cloneNode workflow. Wire it through the `add_node` operation handler and into the persisted node record. The v4 sender will then take the full-copy path on connect without any code change on the v4 side (this is gated entirely on what v5 stores about v4).
- **`add_node` `excludeTables` parameter**: persist on the dynamic node record in the same shape PR #240 reads on static routes. The sender-side skip should then work identically for static and dynamic routes.
- **Default exclusions**: `hdb_info` and `hdb_temp` are already excluded in `replication/databases.ts`. Confirm `hdb_nodes` is the only additional toxic table for v4↔v5. Future cluster-local-state tables should be added to the built-in exclusion list rather than relying on per-route config.

## Schema-compatibility note

v4 → v5 in-place upgrade has historically been supported, so v4's system-table schemas (`hdb_user`, `hdb_role`, etc.) are considered acceptable shape for v5 to ingest. No shape-translation work needed — this issue is only about which tables cross the bridge, not how they're transformed.

## Acceptance criteria

- [ ] `excludeTables` accepted on `sendsTo` / `receivesFrom` route entries (PR #240).
- [ ] `excludeTables` accepted on the `add_node` operation, persisted on the dynamic node record.
- [ ] `isLeader` accepted on the `add_node` operation; setting it triggers full-table-copy from the peer on connect.
- [ ] Sender-side skip works for both static-route and dynamic (`add_node`) configurations.
- [ ] Receiver drops incoming replication messages for excluded tables with a clear log line (not an error — a drop).
- [ ] Integration test (general): two clusters bridged via `add_node { isLeader: true, excludeTables: [{ database: 'system', table: 'hdb_nodes' }] }`. Verify `hdb_user` replicates but `hdb_nodes` stays per-cluster.
- [ ] Integration test (migration): equivalent of `cloneFromLegacy.test.mjs` but using the new bridge primitive. Verify full-copy delivers pre-existing records and the v5 cluster's `hdb_nodes` is unaffected by the v4 source's topology.
- [ ] Runbook §3a rewritten to recommend the new `add_node`-based procedure.

## Related

- [#240](https://github.com/HarperFast/harper-pro/pull/240) — Part A implementation. Needs the dynamic-route gap closed before merge.
- [#237](https://github.com/HarperFast/harper-pro/pull/237) — schema-bootstrap fix; still useful for fresh-install cloneNode with database filtering, no longer load-bearing for migrations.
- [#236](https://github.com/HarperFast/harper-pro/issues/236) — original cloneNode-vs-v4 issue (background).
- [#235](https://github.com/HarperFast/harper-pro/pull/235) — v4↔v5 bridge integration tests. `cloneFromLegacy.test.mjs` now exercises a non-migration path; consider renaming or retiring once the migration test from this issue lands.

🤖 Generated with Claude Opus 4.7.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.