HarperFast / HarperFast/harper-pro
Stress: extend replicationLoad to 500K records + WAN latency + high-freq small-record replication pattern
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 0
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 80
Description
## Context
Current `replicationLoad.test.mjs` uses 5K tiny (~50-byte) records on 3 LAN-local nodes. Production has:
- 1.44M records replicated across 6 nodes (large outdoor retail redirect cluster)
- 7-node MQTT cluster with 428ms max RTT (AP region)
- 856K small connection-event records/day replicated globally across all 7 nodes
## Existing coverage (harper-pro stress suite)
`integrationTests/stress/` already covers several replication patterns:
- `backlogRecovery.test.mjs` — extended node downtime + catch-up, RSS bounded
- `partitionHealConvergence.test.mjs` — split-brain + heal, LWW determinism
- `rapidReconnectAdversity.test.mjs` — rapid connect/disconnect cycles
- `soakWithRollingRestarts.test.mjs` — rolling restarts under sustained writes
## Gaps to fill
### ST-3a: 500K-record convergence with measured convergence time
Extend `replicationLoad` with a large-volume variant:
- 500K inserts across 3 nodes
- Measure time-to-convergence
- Assert convergence within a defined threshold (e.g. 30s for LAN, 120s for simulated WAN)
### ST-3b: Simulated WAN latency (100ms RTT)
Add a `replicationLoad` variant using `tc netem` or equivalent to simulate continental latency:
- Tests backpressure mechanism under real WAN conditions
- Validates that replication doesn't time out or stall under 100ms RTT
### ST-3c: High-frequency small-record replication
Model the MQTT connection-event pattern:
- 1K tiny records/sec written continuously
- Verify they replicate to all peers within a defined window
- Check no growing backlog under steady-state small-record write rate
Contributor guide
Assessment
This issue has not been assessed yet.