tikv / tikv/pd

PD improperly removes orphaned peer without joint consensus, reducing voter count and risking quorum loss

Open
#9,304 4 comments 0 reactions 0 assignees View on GitHub
may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/major type/bug
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Bug Report

In some corner cases, e.g.:
- There are at least two orphan peers
- One of the orphan peers is in pending or down state

PD may schedule a remove-orphaned-peer operator which removes a peer without creating a new one at the same time.

### What did you do?

- Defined a placement rule with higher idx than the default one.
- The rule is, if a store is labeled as jail, offload regions from this store.
- Labeled a store to jail. The store **was not** hosting the region A
- PD scheduled remove-orphaned-peer, made region A under-replicated for a while (2 voters). Until the jail label was removed.

### What did you expect to see?
High availability is guaranteed by maintaining at least 3 voters all the time.

### What did you see instead?
PD removed a voter, causing the region had 2 voters only.

### What version of PD are you using (`pd-server -V`)?
7.5.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.