raft: examine new paper "ReCraft", related to improvements in splits/merges and membership changes

TLDR: In case you haven’t come across this yet, a new paper related to Raft was released very recently. It specifically addresses "improved" mechanisms for splits/merges and membership changes in (multi-)Raft.

The paper includes pseudo-code, formal proofs, implementation code, and benchmarks.

[link to paper](https://arxiv.org/pdf/2504.14802 ).
They also linked the implementation code [here](https://zenodo.org/records/15283088).

The core code is in this path(ReCraft_artifact/ReCraft-Go-Code/raft), which is based on-top of etcd.

They also have the code here on github https://github.com/systems-research-lab/etcd.
But the github repo is incomplete for now.
Also, some statements on CockroachDB in this paper seem to be inaccurate.

Hope it will be an interesting and useful read (if you haven't come across it yet).

---

**Abstract of the paper:**
> ReCraft: Self-Contained Split, Merge, and
> Membership Change of Raft Protocol

> Abstract—Designing reconfiguration schemes for consensus
> protocols is challenging because subtle corner cases during
> reconfiguration could invalidate the correctness of the protocol.
> Thus, most systems that embed consensus protocols conserva-
> tively implement the reconfiguration and refrain from developing
> an efficient scheme. Existing implementations often stop the
> entire system during reconfiguration and rely on a centralized
> coordinator, which can become a single point of failure. We
> present ReCraft, a novel reconfiguration protocol for Raft, which
> supports multi- and single-cluster-level reconfigurations. ReCraft
> does not rely on external coordinators and blocks minimally.
> ReCraft enables the sharding of Raft clusters with split and
> merge reconfigurations and adds a membership change scheme
> that improves Raft. We prove the safety and liveness of ReCraft
> and demonstrate its efficiency through implementations in etcd.
> Index Terms—Consensus protocols, Reconfiguration, Sharding

**Some quotes from the paper:**

first, some jargons used by the paper:
TC - TiKV and CockroachDB
(**the paper seems to combine the 2 entities as 1**, which makes the paper confusing)

CM - coordination manager for splits and merges
AR-RPC - Add/RemoveServer RPC
JC - joint consensus

> Sharding SMR. TiKV [24] and CockroachDB [25] use the
multi-Raft design with logical cluster split and merge features
(Section II-C). Different from ReCraft, their cluster split and
merge operations are driven by the cluster manager, which uses
sophisticated distributed locks and transactions and could be-
come a single point of failure. One may choose to replicate the
cluster manager for fault tolerance (e.g., as in Section VII-D),
but it adds a burden to manage another cluster. ReCraft relies
on the existing consensus mechanism of participating clusters
which is self-contained, more robust, and simple. Note that
ReCraft protocol can be complementarily added to the multi-
Raft design for the logical Raft instance management.

> For both ReCraft and TC,
merging the data takes the longest. Especially, TC exchanges
data with more blocking and can take from 1.7x to 20x more
time than ReCraft depending on the data size. Similar to the
split, the ReCraft approach is more robust and performant.

> ReCraft can tolerate more failures
> for split and the same number for merge compared to TC(TiKV and CockroachDB).
> However, recall that ReCraft does not need separate CMs(coordination manager for splits and merges), is
> more concurrent, and blocks minimally.

> ReCraft performs equal
> to or better than AR-RPC(Add/RemoveServer RP) and JC(joint consensus) for membership changes
> under practical cluster sizes between 2 to 5 except for when
> reducing the cluster size from 5 to 2, which requires one extra
> consensus step than JC.

Jira issue: CRDB-50355