HarperFast / HarperFast/harper-pro
Adaptive replication routing: dynamic fan-out under egress back-pressure
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 0
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 80
Description
## Overview
Dynamically adjust replication routing topology based on back-pressure from egress network limits, switching to fan-out only when needed.
## Design
Under normal conditions, each node replicates directly to all subscribers (lowest latency). When egress limits are hit:
1. Back-pressure detected in outgoing replication threads → reported to main thread
2. Main thread identifies which outgoing connections are back-pressured
3. Signals a child thread to suspend subscription for the highest-latency remote node
4. Child thread sends subscription suspension request; keeps connection open
5. Remote node redirects that subscriber to another node (re-subscribes indirectly)
6. Main thread monitors: resumes direct subscriptions when back-pressure clears
## Key constraint
Harper doesn't proactively push replication — nodes subscribe to receive it. Back-pressure detection must bridge the gap between the sending thread (which sees network limits) and the main thread (which manages topology).
🤖 Filed by Claude on behalf of Kris.
Contributor guide
Assessment
This issue has not been assessed yet.