HarperFast / HarperFast/harper-pro

"Containment mode": explicit operator controls to isolate a node from replication

Open
#169 0 comments 0 reactions 0 assignees View on GitHub
area:replication enhancement from-jira
Dominant language
JavaScript
Stars
3
Forks
0
Avg merge
1d 21h
Merged PRs (30d)
80

Description

**Investigative umbrella.** Provide explicit, operator-facing "containment mode" controls for a Harper node that is in distress — so operators can deliberately isolate a node from replication (stop sending, stop receiving, or both) rather than having to shut it down.

## Context

From the Jira discussion: the underlying capability largely exists today via the `replicates` config flags:

```yaml
replication:
hostname: my-host
routes:
- name: my-host
replicates:
sends: false # don't send out any transactions
receives: true # set false to stop receiving too
```

Per Kris (commented in [CORE-3007](https://harperdb.atlassian.net/browse/CORE-3007)) this should already work after the recent changes to honor `replicates` flags in config, but needs verification.

## Asks

1. **Ergonomic config.** Allow `sends` / `receives` at the **top level** of `replication` config (applied to self), not nested under a route. Easier discovery, fewer ways to misconfigure.
2. **Dynamic toggle via `set_node`.** Allow the same config to be applied to a running instance through `set_node`, including reversing it (an important step). Currently blocked by validation per [CORE-3002](https://harperdb.atlassian.net/browse/CORE-3002).
3. **Cluster-level exile.** Handle the case where the bad node is too broken to respond to its own operations API — needs an `update_node` or equivalent from *other* nodes that tells the cluster to stop waiting on the missing node (today the cluster holds blobs/transactions up to the maximum expiration, potentially filling disk).
4. **Documentation.** Once the above is consolidated, document the containment-mode operator playbook.

## Acceptance criteria

- Operator can isolate a node (sends/receives independently) via config or runtime operation.
- Operator can re-include the node when it's healthy.
- The cluster has a mechanism to stop accumulating undeliverable state for an exiled / unresponsive node.

## Related

- [CORE-3002](https://harperdb.atlassian.net/browse/CORE-3002) — `add_node` should allow `replicates` to be an object with flags (blocker for ask #2).

## Not Ready

Investigation step needed before scheduling: verify ask #1 (`sends`/`receives` config) already works end-to-end, decide whether ask #3 needs a new operation or fits into an existing one (`update_node`, `remove_node` semantics).

---

Tracked in Jira: [CORE-3007](https://harperdb.atlassian.net/browse/CORE-3007)
Jira fixVersion v4.7 is stale — retarget to **V5.2**.

🤖 Filed by Claude on behalf of Kris.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.