etcd-io / etcd-io/etcd

Proposal: Support witness in etcd

Open
#20,696 5 comments 28 reactions 0 assignees View on GitHub
type/feature
Dominant language
Go
Stars
52.3k
Forks
10.5k
Avg merge
2d 21h
Merged PRs (30d)
43

Description

### What would you like to be added?

We propose adding official support for witness storage in etcd clusters, leveraging an extended Raft algorithm to reduce resource consumption while maintaining strong correctness and high availability. In this design, a witness participates in leader election and quorum decisions but does not store the full key-value dataset.

**Key enhancements include:**
- **Modular Raftx Library:** Introduce a decoupled `raftx` library that extends the Raft protocol, enabling witness functionality while preserving a clear separation from the core Raft implementation.
- **Witness-Aware etcd:** Enhance etcd to recognize and interact with witness nodes, allowing lightweight quorum participation and ensuring correctness and availability on par with traditional clusters.
- **Standalone Witness Component:** Implement the witness as an independent component with its own release cycle, ensuring that updates to witness functionality do not impact the stability or cadence of the core etcd project.

This feature aims to deliver a cost-effective solution for achieving high availability, particularly in edge and resource-constrained environments.

A detailed design document is available here: [design doc - etcd with witness](https://docs.google.com/document/d/1JvQQhZ2xiR2qQB74mWv8gRdc2LinzwXgu2l49wwcU4A/edit?usp=sharing)

For more on the extended Raft algorithm, see: [extended Raft algorithm](https://github.com/joshuazh-x/extended-raft-paper/blob/tla/main.pdf)

@ahrtr @serathius @fuweid

### Why is this needed?

etcd is a critical component of Kubernetes, serving as its primary datastore. Ensuring high availability and fault tolerance is essential, but maintaining a full etcd node for every cluster member can be costly, especially for small-sized clusters. By introducing witness nodes, which participate in quorum decisions without replicating the full data set, we can:

- Significantly reduce resource consumption for smaller clusters.
- Lower operational costs by minimizing hardware and storage requirements.
- Enable cost-effective high availability for edge deployments or clusters with limited resources.
- Provide a lightweight alternative for quorum participation without compromising reliability and correctness.

This approach ensures that even small clusters can achieve fault tolerance without incurring the overhead of full etcd nodes.

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked design document and the extended Raft algorithm paper to understand the proposed witness model. Then review how etcd currently handles Raft membership and quorum decisions; done would require an agreed design and implementation of the modular raftx library, witness-aware etcd behavior, and standalone witness component described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.