pingcap / pingcap/tidb

statistics: rework the adaptive mechanism for auto-analyze TiKV scan concurrency

Open
#67,651 1 comment 0 reactions 0 assignees View on GitHub
component/statistics sig/planner type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

# Enhancement

## Summary

The current adaptive mechanism for auto-analyze TiKV scan concurrency looks too aggressive and should not be used as-is.

We should revisit the adaptive formula itself and replace it with a more conservative rule, then run more tests to determine a reasonable long-term solution.

## What we should do

- Do not move forward with the current adaptive mechanism as-is.
- Replace it with a more conservative store-based rule first.
- Re-evaluate the final adaptive strategy after more testing.

## Why the current adaptive mechanism is broken

- The current adaptive mechanism appears too aggressive for larger clusters.
- This parameter directly affects TiKV scan concurrency during auto-analyze, so if we set it too high, we risk consuming too many TiKV resources.
- That means the mechanism may speed up analyze, but it can also create unnecessary pressure on normal workloads.
- In other words, the current formula does not seem to strike the right balance between analyze speed and resource protection.

## Why we need more tests

We still need more data before deciding on the right adaptive rule.

We should test:

- different TiKV store counts
- different table sizes and region counts
- TiKV resource usage under auto-analyze
- the impact on foreground query latency
- whether a direct store-count rule is already good enough, or whether we need a more refined formula

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.