pingcap / pingcap/tidb

Support TopRU

Open
#65,471 1 comment 0 reactions 0 assignees View on GitHub
component/observability feature/accepted type/feature-request
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Feature Request

**Is your feature request related to a problem? Please describe:**

Next-gen TiDB Cloud bills by RU (Request Unit). When cluster RU consumption is abnormal or reaches limits, users need to quickly identify high-RU-consuming SQLs, but there is currently no effective way to identify major RU-consuming SQLs in near real-time.

**Describe the feature you'd like:**

### Goals

1. **Sort by RU Consumption**: Support sorting and querying by cumulative RU consumption to identify high-RU SQLs (including short-execution-time but high-RU SQLs)
2. **User Dimension Aggregation**: Aggregate by `(user, sql_digest, plan_digest)` tuple, supporting per-user RU consumption distribution view
3. **Near Real-time Statistics and (Downstream) Historical Queries**:
- Local 1-second sampling, supporting RU statistics for executing SQLs
- Batch reporting to downstream components (e.g., VM) every 60 seconds, which then write into the storage layer of the observability system
- TiDB only collects and reports RU data; historical querying is provided by the downstream observability system, e.g., querying RU consumption over a recent time window

**Near Real-time Latency Range Definition (End-to-End)**:
- Local sampling written to memory buffer: <= 1s (within one sampling period)
- Observability system visibility (user side): approximately `report_interval (default 60s) + downstream write/query latency`, typically 60~120s
4. **Compatible with Existing Capabilities**: Coexist with TopSQL's existing CPU time statistics without mutual interference

### Non-Goals

1. **TopRU RU ≠ Billing RU**: The RU displayed by TopRU follows the runtime-observed semantics of `util.RUDetails` and is suitable for identifying high-consuming SQLs; billing and auditing should still rely on Billing RU. Aligning these semantics is out of scope for this phase.
2. **Complete Information for Executing SQLs**: Even if an executing SQL has consumed significant RU, complete slow query / SQL statement related information may not be available because the SQL has not yet completed
3. **Anomaly Detection and Auto-Alerting**: Automatic detection of RU consumption anomalies and alert generation is not supported in this phase

### Tasks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.