tikv / tikv/pd

storelimit: synchronize runtime limiters with configuration updates

Open
#11,201 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Enhancement Task

Synchronize configuration updates with the in-memory v1 store limiters in standalone scheduling service deployments, completing the coverage of the earlier fix in #10131 and providing a shared mechanism for AddPeer, RemovePeer, and TransferLeaderIn.

### Background and prior fix

- #10108 reported the same failure pattern on v6.5.7: set the store limit to `0.0001`, consume tokens through add-peer/remove-peer operators, and then raise the limit to `30`. The balance-region scheduler continued filtering out those stores during target selection instead of recovering.
- The [root-cause analysis](https://github.com/tikv/pd/issues/10108#issuecomment-3686032575) identified a circular dependency: the limiter updates its internal rate lazily when handling an operator with a step cost, but the stale, exhausted low-rate limiter prevents target selection from producing such operators.
- #10131, merged into master on January 9, 2026, fixed this for the monolithic PD `RaftCluster.SetStoreLimit` and `SetAllStoresLimit` paths by calling `refreshStoreRateLimit` after configuration updates. Its code and tests were confined to `server/cluster/cluster.go` and `server/cluster/cluster_test.go`.
- #10301 backported that fix to `release-8.5` and was merged on March 9, 2026.

### Remaining gap

The earlier fix did not cover the standalone scheduling service's configuration watch path. That path installs the updated scheduling configuration without immediately refreshing the service's in-memory v1 store limiters.

AddPeer and RemovePeer filters check the existing limiter, while the operator controller refreshes its rate lazily during admission or acknowledgment. After an exhausted, low-rate limit is raised, target selection may therefore continue rejecting stores using the old rate, delaying application of the new configuration.

This issue tracks the remaining MCS coverage gap from #10131 and a shared synchronization mechanism that also covers TransferLeaderIn.

### Expected behavior and validation

Introduce a shared configuration-to-limiter synchronization mechanism for AddPeer, RemovePeer, and TransferLeaderIn, without type-specific filter bypasses or changes to admission semantics.

Validate rate increases, decreases, and transitions to Unlimited through actual scheduler target selection in both monolithic PD and standalone scheduling service deployments. Include the exhausted-low-rate recovery scenario from #10108 so the existing monolithic fix and the standalone scheduling service path are both covered.

Contributor guide

Open the contributing guide

Research direction

Read server/cluster/cluster.go and server/cluster/cluster_test.go to understand the existing refreshStoreRateLimit fix and its validation. Trace the standalone scheduling service configuration watch path and the AddPeer, RemovePeer, and TransferLeaderIn target-selection filters. Done means rate increases, decreases, Unlimited transitions, and exhausted-low-rate recovery work through target selection in both deployment modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.