tikv / tikv/pd

scheduler: balance hot stores formed a lot of cold peers

Open
#5,602 0 comments 0 reactions 0 assignees View on GitHub
type/feature-request
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Feature Request

### Describe your feature request related problem

In some tests, we meet the problem which needs to balance hot stores formed a lot of cold peers.

There are three stores with higher qps than other stores, but the hot scheduler cannot find any hot peers in the three store. Because the store3 has 6k qps, but there are 3k regions in the hot table. In another word, maybe every region is only 2 qps.

| store | qps | hot peer num | hot table leader(without index)|
| :-----| :----: |:----: |:----: |
| 1 | 9k | 2| 4239|
| 2 | 7k | 3| 3405|
| 3 | 6k | 0| 3332|
| 4 | 3k | 2| 3131|
| 5 | 3k | 6| 3049|
| ... | ... | ...| ...|
| 96 | ... | ...|...|
| total | ... | 300| 60000|

20221014171706

We add a scatter region scheduler for only row table to solve the problem as a workaround.

In the future, I hope that PD can solve similar problems by self.

### Describe the feature you'd like

0. we should use more reasonable and unified threshold for hot peers, whatever policy is used. https://github.com/tikv/pd/issues/5155

1. For stores with high qps and less hot peers, we should report topn region to PD. The hot scheduler will solve them. But its speed may be slow, such as there is a store with 4k region that need to schedule.

2. We use a clustering algorithm to identify similar table, and to balance them.

### Teachability, Documentation, Adoption, Migration Strategy

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.