pingcap / pingcap/tidb

Planner: Use column estimation for single column indexes

Open
#67,097 0 comments 0 reactions 0 assignees View on GitHub
sig/planner type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

Column predicate selectivity estimation can be recalculated multiple times. And also, a single column index range (for a single column index or multi-column index) will also be recalculated using index statistics after it was also calculated at the column level.

Here's an example of the full duplication chain for a single-column index with a column predicate like WHERE a > 10:

What happens today

1. Selectivity() line 124/132 — calls GetRowCountByColumnRanges(col_a, ranges) → builds a ColType StatsNode
2. Selectivity() line 192 — calls GetRowCountByIndexRanges(idx_a, ranges) → builds an IndexType StatsNode. Inside this:
- getIndexRowCountForStatsV2 does betweenRowCountOnIndex (string-encoded histogram) for in-range
- Then uses c.Histogram.OutOfRangeRowCount (column histogram, original types) for out-of-range
3. deriveTablePathStats() line 345 — calls GetRowCountByColumnRanges(pk, ranges) again for the PK path
4. crossEstimateRowCount() — may re-estimate the same column with different ranges

For a single-column index, step 2 produces an estimate that's worse than step 1 — it uses a lossy string-encoded histogram for in-range, then patches with the column histogram for out-of-range. Step 1 uses the column histogram (original types) for everything.

TiDB should calculate the individual column estimate once from the column statistics, and reuse that in all estimation whenever that single column is used (including as a single column range in an index accesscond).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.