pingcap / pingcap/tidb

Improve the method of estimating the table scan tasks for index lookup

Open
#43,503 0 comments 0 reactions 0 assignees View on GitHub
sig/planner type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

Currently, we use a fixed magic factor to estimate how many table scan tasks there will be for the row IDs read from the index.

However, this highly depends on the data distribution. Specifically, that means, when we read a range from the index, is the row IDs more clustered or more scattered in the table. If it's more clustered, the number of table scan tasks will be small, and the performance of index lookup will be good, and vice versa.

Some other databases have been using some statistics to solve such problems, like correlation factor and clustering factor. We can also learn from them.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.