cockroachdb / cockroachdb/cockroach
raft: pre-candidate can spuriously win election from vote in prior term
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
From https://github.com/cockroachdb/cockroach/pull/123189#issuecomment-2083427479, see the **PreVote bug**:
> I believe that there is a bug in the existing PreVote algorithm in etcd/raft (and our new fork) that can allow a pre-vote candidate to win an pre-vote election that it otherwise shouldn't. Specifically, pre-vote elections are called with the [term following the current term](https://github.com/etcd-io/raft/blob/5a610f60542af2b7064a6bdfa6f7793a9b00c2f1/raft.go#L1037), which is carried by MsgPreVote messages. This term is also carried in [non-rejected MsgPreVoteResp messages](https://github.com/etcd-io/raft/blob/5a610f60542af2b7064a6bdfa6f7793a9b00c2f1/raft.go#L1244). Pre-vote candidates [know to handle these](https://github.com/etcd-io/raft/blob/5a610f60542af2b7064a6bdfa6f7793a9b00c2f1/raft.go#L1109) non-rejected MsgPreVoteResp messages with a future term. However, they have no special handling of non-rejected MsgPreVoteResp messages with their current term. These messages correspond to a pre-vote election held at the previous term, but it looks like they can be used to win a pre-election at the current term.
>
> For example, I think the following order of operations is allowed:
>
> ```
> 1. n3 is the leader at term 10
> 2. n1 calls a pre-vote election at term 10, sends MsgPreVote{Term: 11} to n2 and n3
> 3. n2 responds with a successful MsgPreVoteResp{Term: 11, Reject: false}, but the message is delayed
> 4. n1 never wins the pre-election
> 5. eventually the term advances to term 11 and n2 takes over as leader
> 6. n1 calls a pre-vote election at term 11, sends MsgPreVote{Term: 12} to n2 and n3
> 7. n2 and n3 both reject these, however, before they can respond...
> 8. the original message from n3 is delivered, n1 thinks it won the pre-vote election
> 9. n1 calls a disruptive election for term 12
> ```
>
> The consequences for this bug are not overly severe, because a pre-vote election is a best-effort attempt to prevent a disruptive leader election. Bugs in pre-vote do not compromise safety (i.e. leader exclusivity). Still, we should fix this.
Jira issue: CRDB-38276
Contributor guide
Assessment
This issue has not been assessed yet.