PD improperly removes orphaned peer without joint consensus, reducing voter count and risking quorum loss
- Dominant language
- Go
- Stars
- 1.2k
- Forks
- 783
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 36
Description
## Bug Report
In some corner cases, e.g.:
- There are at least two orphan peers
- One of the orphan peers is in pending or down state
PD may schedule a remove-orphaned-peer operator which removes a peer without creating a new one at the same time.
### What did you do?
- Defined a placement rule with higher idx than the default one.
- The rule is, if a store is labeled as jail, offload regions from this store.
- Labeled a store to jail. The store **was not** hosting the region A
- PD scheduled remove-orphaned-peer, made region A under-replicated for a while (2 voters). Until the jail label was removed.
### What did you expect to see?
High availability is guaranteed by maintaining at least 3 voters all the time.
### What did you see instead?
PD removed a voter, causing the region had 2 voters only.
### What version of PD are you using (`pd-server -V`)?
7.5.1
Contributor guide
Assessment
This issue has not been assessed yet.