planner: ORDER BY LIMIT regresses from TiKV IndexLookUp in v7.5.6 to TiFlash MPP on master
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
Replayer name: `replayer_9UxgMSowmgw-mErjFWHg6Q==_1773212242351066908.zip`
I can reproduce the plan change with the same replayed stats/settings on both versions.
1. Start a cluster with TiFlash:
```bash
tiup playground v7.5.6 --tiflash=1
```
2. Load the replayer:
```sql
PLAN REPLAYER LOAD 'replayer_9UxgMSowmgw-mErjFWHg6Q==_1773212242351066908.zip';
```
3. Run the anonymized query below (same shape as `sql/sql0.sql` in the replayer):
```sql
SELECT /* anonymized */
...
FROM db_x.t_x
WHERE order_col >= '2026-01-02 00:00:01'
AND filter_col_1 = 3
AND filter_col_2 = 1
AND filter_col_3 = 0
AND filter_col_4 >= '2026-01-01 00:00:01'
ORDER BY order_col
LIMIT 10;
```
4. `EXPLAIN` on `v7.5.6` picks a TiKV ordered `IndexLookUp`.
5. Repeat with `master` / nightly (I reproduced on local master `a5088d32182f42f594dee37108ab10373e04b1d7`), with the same replayed stats/settings and TiFlash available. `EXPLAIN` switches to a TiFlash MPP `TopN + TableFullScan` plan.
Representative anonymized plan shapes:
`v7.5.6` or `set @@tidb_opt_ordering_index_selectivity_ratio = 0.0001`
```text
+--------------------------+---------+-----------+-------------------------------------------+
| id | estRows | task | operator info |
+--------------------------+---------+-----------+-------------------------------------------+
| Limit | 10.00 | root | offset:0, count:10 |
| └─IndexLookUp | 10.00 | root | |
| ├─Selection(Build) | 8316.60 | cop[tikv] | eq(filter_col_1, 3) |
| │ └─IndexRangeScan | 8316.60 | cop[tikv] | range:[2026-01-02 00:00:01,+inf], ordered |
| └─Selection(Probe) | 10.00 | cop[tikv] | other residual filters |
| └─TableRowIDScan | 8316.60 | cop[tikv] | |
+--------------------------+---------+-----------+-------------------------------------------+
```
`master` default
```text
+--------------------------+-----------+--------------+------------------------------------+
| id | estRows | task | operator info |
+--------------------------+-----------+--------------+------------------------------------+
| TopN | 10.00 | root | order by order_col, count:10 |
| └─TableReader | 10.00 | root | MppVersion: 3 |
| └─ExchangeSender | 10.00 | mpp[tiflash] | ExchangeType: PassThrough |
| └─TopN | 10.00 | mpp[tiflash] | order by order_col, count:10 |
| └─Selection | 68575.11 | mpp[tiflash] | all filters |
| └─TableFullScan | 152304.43 | mpp[tiflash] | keep order:false |
+--------------------------+-----------+--------------+------------------------------------+
```
### 2. What did you expect to see? (Required)
I expected `master` to still choose the TiKV ordered index path for this query shape, similar to `v7.5.6`, because the query is `ORDER BY ... LIMIT 10` and there is an available ordered index path.
### 3. What did you see instead (Required)
`master` only chooses the TiFlash MPP plan for this case.
After local bisect, the first bad commit appears to be:
- `22bd51eed0f60e94f327c1e33f9bb08940f4bab9`
- `planner: adjust index scan estimate for order with limit (#50345)`
The regression seems to come from the ordered-index penalty added in `AdjustRowCountForIndexScanByLimit` together with the default `tidb_opt_ordering_index_selectivity_ratio = 0.01`.
In this replayed case, `CountAfterAccess` is very large (stats row count is tens of millions), so:
```text
rowsToMeetFirst := (path.CountAfterAccess - rowCount) * 0.01
rowCount += rowsToMeetFirst
```
adds hundreds of thousands of rows to the ordered `IndexLookUp` path estimate. That is enough to make the TiKV ordered index path more expensive than the TiFlash MPP full scan.
I also verified that lowering or disabling this heuristic restores the TiKV plan:
- `set @@tidb_opt_ordering_index_selectivity_ratio = 0.0001;` -> TiKV `IndexLookUp`
- `set @@tidb_opt_ordering_index_selectivity_ratio = -1;` -> TiKV `IndexLookUp`
So this looks like the heuristic may be too aggressive for large `CountAfterAccess` cases, especially when the index is still useful for both ordering and part of the filtering.
### 4. What is your TiDB version? (Required)
- `v7.5.6`: reproducibly chooses TiKV `IndexLookUp`
- `master` (`a5088d32182f42f594dee37108ab10373e04b1d7`, local build on 2026-03-12): reproducibly chooses TiFlash MPP `TopN + TableFullScan`
### Additional context
Two possible fix directions that seem worth checking:
1. Add stronger guards before applying `rowsToMeetFirst`, for example only when the path is really "order-first but filter-late".
2. Add a cap to `rowsToMeetFirst`, so a very large `CountAfterAccess` does not completely dominate the ordered index path cost.
I do not think this should be solved by globally raising TiFlash cost, because the issue seems to be the ordered-index penalty being too large in this specific class of queries.
Contributor guide
Assessment
This issue has not been assessed yet.