cockroachdb / cockroachdb/cockroach

raft: faster check-quorum step-down with leader leases

Open
#144,478 0 comments 0 reactions 0 assignees View on GitHub
A-kv-replication A-leader-leases C-enhancement T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Currently, when a leader steps down due to loss of quorum, it takes ~7s in the worst case: 3s for store liveness support for the leader to expire, 2 seconds for the first check quorum after that to mark the followers as inactive, and 2s for the second check quorum to confirm that the followers are still inactive.

There is no specific reason we need to wait this long with leader leases. This is mostly an artifact of the existing `CheckQuorum` implementation and the use of the `RecentActive` field to detect inactive followers:

https://github.com/cockroachdb/cockroach/blob/720709b6c8721a10c9bdac31cb7b51f81c628f17/pkg/raft/tracker/progress.go#L105-L111

We should aim for a leader step-down time closer to the store liveness support expiry (3s), which is already a strong enough signal.

Jira issue: CRDB-49392

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.