PD should evict all leaders from the deleted store before moving replicas aggressively
- Dominant language
- Go
- Stars
- 1.2k
- Forks
- 783
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 36
Description
## Bug Report
### What did you do?
Delete a store with the default settings.
### What did you expect to see?
The cluster continue to operate smoothly with slightly increasing latency.
### What did you see instead?
The cluster has a lot of jitters.
When deleting a store, PD will set remove peer limit to 100000000, which means the speed depends on the number of available up stores. When deleting a lot of ranges in TiKV, it will change the LSM tree rapidly and causing a lot of compaction, which will consume both CPU and IO. If there are still leaders on the deleted store, the latency can be very high and unstable.

I added an evict leader scheduler manually in the left part, and remove it in the right part. The latency and throughput is changed dramatically.

### What version of PD are you using (`pd-server -V`)?
```
Release Version: v5.1.1
Edition: Community
Git Commit Hash: 7cba1912b317a533e18b16ea2ba9a14ed2891129
Git Branch: heads/refs/tags/v5.1.1
UTC Build Time: 2021-07-23 01:16:18
```
Contributor guide
Assessment
This issue has not been assessed yet.