pingcap / pingcap/tidb

Statistics Tech Debt

Open
#55,043 1 comment 2 reactions 1 assignee Claimed by @0xPoe View on GitHub
component/statistics sig/planner type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Tech Debt

In this issue, I will record all tech debts I found in the TiDB statistics module.

## Code duplication

The worst and most serious technical debt in the statistics module is code redundancy, and the following implementation pattern is used almost everywhere there is concurrent processing:
1. Keep single-threaded code untouched.
2. Directly implement another concurrent version of the exact same code.

This pattern creates a lot of problems, we have a few issues that need to be fixed twice, and code redundancy is serious. Theoretically, we just need to think of single-threaded as a special case of a multi-threaded implementation, and we shouldn't be copying and pasting code.

Here are the relevant modules that have this problem:
1. Result Stored Procedures for Partitioned Tables
2. Init Stats
3. Merge Global Stats
and so on...

## Too many variables

[image
](https://tidb-analyze.vercel.app/44?clicks=3)

We have a lot of variables related to the collection of statistics, most of which were introduced when concurrency support was introduced, and they have a variety of names that are very difficult to understand. It is also not clear from the documentation how these variables affect the system.
Even some of these variables are actually related to each other, which makes it very challenging for users to work with statistical information.
We need to:
1. check that these variables are actually in effect.
2. explain how these variables affect the system.
3. ensure that this information can be found in the documentation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.