apache / apache/kvrocks-controller

Implement Multi-node Quorum-based Health Checking for Kvrocks Nodes

Open
#392 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
141
Forks
76
PR merge metrics
No merged PRs in 30d

Description

**Motivation**
In a multi-controller deployment, triggering failover based strictly on a single controller's local probes is risky. If that controller experiences partial network degradation to a Kvrocks node — while its peers observe a healthy state — it may initiate an unwarranted failover, leading to a split-brain scenario and unnecessary traffic shedding.

**Implementation**

- Voting mechanism: Each controller independently probes every Kvrocks node. Before promoting a new master, the leader collects **unanimous votes** from all peer controllers via `POST /internal/vote`. A single NO (or unreachable peer with a live lease) blocks the failover.

- Peer discovery: Controllers register themselves in the store with a heartbeat. `ListActivePeers` returns peers whose leases are still alive. Expired peers are excluded from quorum rather than blocking failover.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.