pingcap / pingcap/tidb

Enhance the strategy of choosing which table to run auto analyze

Open
#34,207 0 comments 0 reactions 1 assignee Claimed by @hawkingrei View on GitHub
type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

Currently, `autoAnalyzeWorker` checks each table in a fixed order every 3 seconds and finds the first need-analyze table to trigger one auto analyze job. If the auto analyze job of some table fails for some reason, `autoAnalyzeWorker` will choose the same table to analyze next time and fail again. In this way, failure of analyzing one table can block auto analyze of other tables. Randomizing the order can fix the problem. Further, we can design a priority queue to place the table which needs analyze most in the front. The priority may depend on several factors:
1. the table stats healthy(including modify_count / count and whether stats of some column or index are missing)
2. how frequently the table stats are used
3. time cost of analyzing the table
4. whether the auto analyze job of the table failed before
5. ...

Hence we split the enhancement into two parts.
- [ ] randomize the order as a quick fix
- [ ] design a priority queue to place the table which needs analyze most in the front

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.