pingcap / pingcap/tidb

planner: ORDER BY LIMIT regresses from TiKV IndexLookUp in v7.5.6 to TiFlash MPP on master

Open
#66,930 0 comments 0 reactions 0 assignees View on GitHub
AI-Correction severity/moderate sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

Replayer name: `replayer_9UxgMSowmgw-mErjFWHg6Q==_1773212242351066908.zip`

I can reproduce the plan change with the same replayed stats/settings on both versions.

1. Start a cluster with TiFlash:

```bash
tiup playground v7.5.6 --tiflash=1
```

2. Load the replayer:

```sql
PLAN REPLAYER LOAD 'replayer_9UxgMSowmgw-mErjFWHg6Q==_1773212242351066908.zip';
```

3. Run the anonymized query below (same shape as `sql/sql0.sql` in the replayer):

```sql
SELECT /* anonymized */
...
FROM db_x.t_x
WHERE order_col >= '2026-01-02 00:00:01'
AND filter_col_1 = 3
AND filter_col_2 = 1
AND filter_col_3 = 0
AND filter_col_4 >= '2026-01-01 00:00:01'
ORDER BY order_col
LIMIT 10;
```

4. `EXPLAIN` on `v7.5.6` picks a TiKV ordered `IndexLookUp`.

5. Repeat with `master` / nightly (I reproduced on local master `a5088d32182f42f594dee37108ab10373e04b1d7`), with the same replayed stats/settings and TiFlash available. `EXPLAIN` switches to a TiFlash MPP `TopN + TableFullScan` plan.

Representative anonymized plan shapes:

`v7.5.6` or `set @@tidb_opt_ordering_index_selectivity_ratio = 0.0001`

```text
+--------------------------+---------+-----------+-------------------------------------------+
| id | estRows | task | operator info |
+--------------------------+---------+-----------+-------------------------------------------+
| Limit | 10.00 | root | offset:0, count:10 |
| └─IndexLookUp | 10.00 | root | |
| ├─Selection(Build) | 8316.60 | cop[tikv] | eq(filter_col_1, 3) |
| │ └─IndexRangeScan | 8316.60 | cop[tikv] | range:[2026-01-02 00:00:01,+inf], ordered |
| └─Selection(Probe) | 10.00 | cop[tikv] | other residual filters |
| └─TableRowIDScan | 8316.60 | cop[tikv] | |
+--------------------------+---------+-----------+-------------------------------------------+
```

`master` default

```text
+--------------------------+-----------+--------------+------------------------------------+
| id | estRows | task | operator info |
+--------------------------+-----------+--------------+------------------------------------+
| TopN | 10.00 | root | order by order_col, count:10 |
| └─TableReader | 10.00 | root | MppVersion: 3 |
| └─ExchangeSender | 10.00 | mpp[tiflash] | ExchangeType: PassThrough |
| └─TopN | 10.00 | mpp[tiflash] | order by order_col, count:10 |
| └─Selection | 68575.11 | mpp[tiflash] | all filters |
| └─TableFullScan | 152304.43 | mpp[tiflash] | keep order:false |
+--------------------------+-----------+--------------+------------------------------------+
```

### 2. What did you expect to see? (Required)

I expected `master` to still choose the TiKV ordered index path for this query shape, similar to `v7.5.6`, because the query is `ORDER BY ... LIMIT 10` and there is an available ordered index path.

### 3. What did you see instead (Required)

`master` only chooses the TiFlash MPP plan for this case.

After local bisect, the first bad commit appears to be:

- `22bd51eed0f60e94f327c1e33f9bb08940f4bab9`
- `planner: adjust index scan estimate for order with limit (#50345)`

The regression seems to come from the ordered-index penalty added in `AdjustRowCountForIndexScanByLimit` together with the default `tidb_opt_ordering_index_selectivity_ratio = 0.01`.

In this replayed case, `CountAfterAccess` is very large (stats row count is tens of millions), so:

```text
rowsToMeetFirst := (path.CountAfterAccess - rowCount) * 0.01
rowCount += rowsToMeetFirst
```

adds hundreds of thousands of rows to the ordered `IndexLookUp` path estimate. That is enough to make the TiKV ordered index path more expensive than the TiFlash MPP full scan.

I also verified that lowering or disabling this heuristic restores the TiKV plan:

- `set @@tidb_opt_ordering_index_selectivity_ratio = 0.0001;` -> TiKV `IndexLookUp`
- `set @@tidb_opt_ordering_index_selectivity_ratio = -1;` -> TiKV `IndexLookUp`

So this looks like the heuristic may be too aggressive for large `CountAfterAccess` cases, especially when the index is still useful for both ordering and part of the filtering.

### 4. What is your TiDB version? (Required)

- `v7.5.6`: reproducibly chooses TiKV `IndexLookUp`
- `master` (`a5088d32182f42f594dee37108ab10373e04b1d7`, local build on 2026-03-12): reproducibly chooses TiFlash MPP `TopN + TableFullScan`

### Additional context

Two possible fix directions that seem worth checking:

1. Add stronger guards before applying `rowsToMeetFirst`, for example only when the path is really "order-first but filter-late".
2. Add a cap to `rowsToMeetFirst`, so a very large `CountAfterAccess` does not completely dominate the ordered index path cost.

I do not think this should be solved by globally raising TiFlash cost, because the issue seems to be the ordered-index penalty being too large in this specific class of queries.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.