tikv / tikv/pd

Speed up multi-replica TiFlash placement in large clusters

Open
#11,144 1 comment 0 reactions 0 assignees View on GitHub
type/feature-request
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Feature Request

### Describe your feature request related problem

When a placement rule requires multiple TiFlash learners, RuleChecker creates one `add-rule-peer` operator per check. After the operator completes, the Region waits for a later patrol before RuleChecker discovers and schedules the next missing learner.

This delay can be significant in clusters with many Regions, slowing down multi-replica TiFlash placement.

It can also delay Scatter because PD requires the placement rules to be satisfied first. TiDB PR [[#68402](https://github.com/pingcap/tidb/pull/68402)](https://github.com/pingcap/tidb/pull/68402) mitigates this from the DDL side by waiting for TiFlash placement before scattering newly created ranges.

### Describe the feature you'd like

After a RuleChecker-generated `add-rule-peer` operator succeeds, prioritize checking the same Region again instead of waiting for the next patrol.

If the placement rules are still not satisfied, RuleChecker can schedule the next operator when scheduling limits allow. Peer changes remain serialized.

### Describe alternatives you've considered

- Continue relying on normal patrol, which can introduce a long delay in large clusters.
- Configure TiFlash replicas incrementally so each placement rule update triggers another prioritized check.
- Recheck the entire affected key range, which may perform unnecessary work compared with rechecking the completed Region.

### Teachability, Documentation, Adoption, Migration Strategy

This is an internal scheduling improvement. It should not require API changes, new configuration, or user migration.

Contributor guide

Open the contributing guide

Research direction

Start by tracing RuleChecker's handling of completed add-rule-peer operators and the patrol path that checks Regions. Verify how scheduling limits and serialized peer changes are enforced. Done means a successful operator prioritizes the same Region for another check, allowing the next missing learner to be scheduled without waiting for a later patrol.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.