planner: correlated equality is not considered a constant index prefix for order property matching
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step (Required)
Create a table with a composite index whose first column is constrained by a correlated equality and whose second column provides the required order:
```sql
CREATE TABLE accounts (
id BIGINT PRIMARY KEY,
user_id BIGINT,
KEY idx_user_id(user_id)
);
CREATE TABLE position_snapshots (
id BIGINT PRIMARY KEY,
account_id BIGINT NOT NULL,
snapshot_at DATETIME(6) NOT NULL,
payload JSON,
KEY idx_position_snapshots_snapshot_at(account_id, snapshot_at)
);
EXPLAIN
SELECT a.id,
(
SELECT ps.payload
FROM position_snapshots ps
WHERE ps.account_id = a.id
ORDER BY ps.snapshot_at DESC
LIMIT 1
) AS latest_snapshot
FROM accounts a
WHERE a.user_id = 8295;
```
In the observed plan, the inner side of `Apply` uses an unordered lookup followed by a two-stage TopN:
```text
Apply
└─TopN(root) snapshot_at:desc, count:1
└─IndexLookUp
├─IndexRangeScan account_id = CorrelatedColumn(a.id), keep order:false
└─TopN(cop) snapshot_at:desc, count:1
└─TableRowIDScan
```
The issue is reproducible even though an index on `(account_id, snapshot_at)` is available.
### 2. What did you expect to see? (Required)
For each `Apply` probe, `a.id` is a runtime constant. Therefore:
```sql
ps.account_id = a.id
```
fixes the first index column for that probe, and `(account_id, snapshot_at)` can provide `snapshot_at DESC` by scanning the matching index range in reverse order.
The optimizer should be able to enumerate an ordered inner path similar to:
```text
Apply
└─Limit 1
└─IndexLookUp
└─IndexRangeScan(account_id = CorrelatedColumn(a.id))
keep order:true, desc:true
```
The logical TopN already enumerates a `PhysicalLimit` alternative that requests the `snapshot_at DESC` property. The composite index should be able to satisfy that property because its first column is fixed by the correlated equality for each probe.
### 3. What did you see instead (Required)
The `PhysicalLimit` alternative is rejected during access-path property matching, leaving only the unordered path plus TopN.
The planner already treats `col = CorrelatedColumn` as a runtime-constant equality for range construction:
1. `SplitCorColAccessCondFromFilters` moves `index_col = correlated_col` from table filters into `AccessConds`.
2. `ResolveCorrelatedColumns` rebuilds the range using the current outer-row value during execution.
However, after the correlated condition is appended to `AccessConds`, its constant-prefix property is not propagated to the metadata used by `matchProperty`. In particular, `AccessPath.ConstCols` is populated only from `DetachRangeResult.ColumnValues`, which represents values available during static range detachment.
`matchProperty` can skip an index prefix only when either:
- `path.ConstCols[colIdx]` is true, or
- the statically built ranges prove that the index column is a point value.
A correlated equality satisfies neither check at planning time, even though it is fixed for each `Apply` probe. Consequently, `(account_id, snapshot_at)` is incorrectly considered unable to satisfy `ORDER BY snapshot_at DESC`.
Relevant code paths:
- `pkg/planner/util/path.go`: `SplitCorColAccessCondFromFilters`
- `pkg/planner/core/stats.go`: `deriveIndexPathStats` and `detachCondAndBuildRangeForPath`
- `pkg/planner/core/find_best_task.go`: `matchProperty`
The key issue is that the runtime-constant nature of the correlated access condition is used for range construction but not for order-property matching.
### 4. What is your TiDB version? (Required)
```text
Release Version: v8.5.6
Git Commit Hash: ae18096e023780bb56bfce33698abec0d4640d0a
```
Contributor guide
Assessment
This issue has not been assessed yet.