storelimit: synchronize runtime limiters with configuration updates
- Dominant language
- Go
- Stars
- 1.2k
- Forks
- 783
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 36
Description
## Enhancement Task
Synchronize configuration updates with the in-memory v1 store limiters in standalone scheduling service deployments, completing the coverage of the earlier fix in #10131 and providing a shared mechanism for AddPeer, RemovePeer, and TransferLeaderIn.
### Background and prior fix
- #10108 reported the same failure pattern on v6.5.7: set the store limit to `0.0001`, consume tokens through add-peer/remove-peer operators, and then raise the limit to `30`. The balance-region scheduler continued filtering out those stores during target selection instead of recovering.
- The [root-cause analysis](https://github.com/tikv/pd/issues/10108#issuecomment-3686032575) identified a circular dependency: the limiter updates its internal rate lazily when handling an operator with a step cost, but the stale, exhausted low-rate limiter prevents target selection from producing such operators.
- #10131, merged into master on January 9, 2026, fixed this for the monolithic PD `RaftCluster.SetStoreLimit` and `SetAllStoresLimit` paths by calling `refreshStoreRateLimit` after configuration updates. Its code and tests were confined to `server/cluster/cluster.go` and `server/cluster/cluster_test.go`.
- #10301 backported that fix to `release-8.5` and was merged on March 9, 2026.
### Remaining gap
The earlier fix did not cover the standalone scheduling service's configuration watch path. That path installs the updated scheduling configuration without immediately refreshing the service's in-memory v1 store limiters.
AddPeer and RemovePeer filters check the existing limiter, while the operator controller refreshes its rate lazily during admission or acknowledgment. After an exhausted, low-rate limit is raised, target selection may therefore continue rejecting stores using the old rate, delaying application of the new configuration.
This issue tracks the remaining MCS coverage gap from #10131 and a shared synchronization mechanism that also covers TransferLeaderIn.
### Expected behavior and validation
Introduce a shared configuration-to-limiter synchronization mechanism for AddPeer, RemovePeer, and TransferLeaderIn, without type-specific filter bypasses or changes to admission semantics.
Validate rate increases, decreases, and transitions to Unlimited through actual scheduler target selection in both monolithic PD and standalone scheduling service deployments. Include the exhausted-low-rate recovery scenario from #10108 so the existing monolithic fix and the standalone scheduling service path are both covered.
Contributor guide
Research direction
Read server/cluster/cluster.go and server/cluster/cluster_test.go to understand the existing refreshStoreRateLimit fix and its validation. Trace the standalone scheduling service configuration watch path and the AddPeer, RemovePeer, and TransferLeaderIn target-selection filters. Done means rate increases, decreases, Unlimited transitions, and exhausted-low-rate recovery work through target selection in both deployment modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100