tikv / tikv/pd

resource control: align SQL layer RU accounting with RUMode limiting semantics

Open
#10,940 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Enhancement Task

### Problem

Resource Control currently uses different RU scopes for token limiting and metrics:

- In the client controller, RUMode limiting uses `getRUValueFromConsumption()`, which returns only `RRU + WRU`.
- `SQLCalculator.Trickle()` records SQL layer CPU into `TotalCpuTimeMs` and `SqlLayerCpuTimeMs`.
- The server converts SQL layer CPU to RU through `SqlLayerCpuTimeMs * CPUMsCost` and includes it in the V1 active/consumed RU metric.

As a result, metrics that represent total consumed RU can include SQL layer CPU RU, while the local token bucket and the available RU/service-limit view are driven by KV read/write RU only. For SQL-layer-heavy workloads, consumed RU can be much higher than the configured fill rate while available RU does not deplete correspondingly. This is confusing because the metrics look like they describe the same budget, but they do not.

https://github.com/tikv/pd/blob/2e12f481cd492abef45a7b6af59ea6649ceaf268/client/resource_group/controller/model.go#L217-L222

### Expected improvement

We should make this behavior explicit and internally consistent. There are two possible directions:

1. If the configured fill rate is intended to cap total RU, include SQL layer CPU RU in RUMode token demand/accounting. The design needs to avoid double counting with existing RRU/WRU and account for the fact that SQL CPU is sampled from process CPU deltas.
2. If the configured fill rate is intentionally scoped to KV read/write RU only, keep the limiter behavior but clarify observability: separate the "limited RU" scope from the "total consumed RU" scope in metric names, dashboard usage, or documentation, and expose any missing metric needed to compare limiter demand with available RU.

### Code references

- `client/resource_group/controller/model.go`
- `SQLCalculator.Trickle()` updates `TotalCpuTimeMs` and `SqlLayerCpuTimeMs`.
- `getRUValueFromConsumption()` returns only `RRU + WRU`.
- `client/resource_group/controller/group_controller.go`
- `calcAvg()`, `acquireTokens()`, and `onResponseImpl()` use `getRUValueFromConsumption()` for RUMode token accounting.
- `pkg/mcs/resourcemanager/server/metrics.go`
- `calculateSQLRU()` converts SQL CPU to RU.
- `calculateActiveRU()` includes SQL layer RU for V1 keyspaces.
- `pkg/mcs/resourcemanager/server/grpc_service.go`
- token requests are based on client-reported token demand, not on the server-side SQL RU metric.

### Background

The current behavior appears to be long-standing:

- RUMode token accounting was introduced using read/write RU as the limiter scope.
- SQL layer CPU RU was later added as statistics/metrics.
- The later active RU metric intentionally includes SQL layer RU for V1 metrics, but it does not change the limiter path.

This issue tracks a design/implementation decision rather than a confirmed regression.

### Related

- #10488 tracks broader Resource Control observability improvements.
- #10581 tracks a client-side pre-throttling RU/s demand metric.

Contributor guide

Open the contributing guide

Research direction

Start with getRUValueFromConsumption(), SQLCalculator.Trickle(), calcAvg(), acquireTokens(), and onResponseImpl() in the client resource-group controller, then compare them with calculateSQLRU() and calculateActiveRU() in server/metrics.go and token handling in grpc_service.go. Choose and document one RU scope, then verify that token demand, available RU, and reported metrics consistently reflect that decision without double counting.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.