pingcap / pingcap/tidb

planner: tidb_opt_enable_no_decorrelate_in_select=1 returns wrong results for outer aggregates in SELECT-list subqueries

Open
#70,286 0 comments 0 reactions 0 assignees View on GitHub
affects-8.5 impact/wrong-result may-affects-7.5 may-affects-8.1 severity/critical sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

**Affected versions**: master (since #63204) and release-8.5 v8.5.4+ (cherry-pick #63541). Found during a PR-by-PR review of release-8.5. The variable `tidb_opt_enable_no_decorrelate_in_select` defaults to OFF, so this only triggers when it is explicitly enabled.

### 1. Minimal reproduce step (Required)

```sql
DROP DATABASE IF EXISTS d;
CREATE DATABASE d;

CREATE TABLE d.t1 (a INT, b INT);
CREATE TABLE d.t2 (a INT, b INT);
INSERT INTO d.t1 VALUES (1, 10), (1, 20), (2, 30);
INSERT INTO d.t2 VALUES (1, 1), (1, 2), (2, 3);

SET SESSION tidb_opt_enable_no_decorrelate_in_select = 1;

SELECT a, SUM(t1.b) AS total,
(SELECT SUM(t2.b) + SUM(t1.b) FROM d.t2 WHERE t2.a = t1.a) AS v
FROM d.t1 AS t1
GROUP BY a
ORDER BY a;
```

### 2. What did you expect to see? (Required)

The result must not depend on the optimization variable. With the variable OFF (and in MySQL semantics):

```
a | total | v
1 | 30 | 33 -- SUM(t2.b)=3 for a=1, plus SUM(t1.b)=30 for group a=1
2 | 30 | 33 -- SUM(t2.b)=3 for a=2, plus SUM(t1.b)=30 for group a=2
```

`SUM(t1.b)` inside the subquery references only the outer query's group, so it is an outer aggregate that must be evaluated in the outer grouping context.

### 3. What did you see instead (Required)

With `tidb_opt_enable_no_decorrelate_in_select = 1`:

```
a | total | v
1 | 30 | 23 -- wrong: 20 + 3
2 | 30 | 33
```

The outer aggregate `SUM(t1.b)` is computed per correlated row inside the subquery instead of over the outer group (for `a=1` it picks up 20 instead of 30). Setting the variable back to 0 returns the correct result.

### 4. What is your TiDB version? (Required)

Reproduced on v8.5.7 (release-8.5, git 1fdc13626a). Root cause confirmed by code inspection in master @ 6f5bfe198f (2026-08-01), which carries identical code.

### Root cause analysis

#63204 added `tidb_opt_enable_no_decorrelate_in_select`; `PlanBuilder.Init` sets `b.noDecorrelate = sctx.GetSessionVars().EnableNoDecorrelateInSelect` (`pkg/planner/core/planbuilder.go:506`). In `extractCorrelatedAggFuncs` (`pkg/planner/core/logical_plan_builder.go:3140-3143` on release-8.5, `:3041` on master):

```go
// If decorrelation is disabled, don't extract correlated aggregates
if b.noDecorrelate && len(corCols) > 0 {
continue
}
if len(corCols) > 0 && len(cols) == 0 {
outer = append(outer, agg)
}
```

For `SUM(t2.b) + SUM(t1.b)`, `SUM(t1.b)` has `corCols > 0` and `cols == 0` — it depends only on outer columns and must be hoisted into the outer aggregation. With no-decorrelate enabled the `continue` skips that extraction, so the aggregate stays in the subquery's aggregate list and is evaluated per correlated outer row, producing wrong results. (A secondary issue in the same hunk: `continue` also skips the `corCols, cols = corCols[:0], cols[:0]` reset, so subsequent aggregates inherit stale correlated columns.)

Suggested direction: when `noDecorrelate` is set, still hoist aggregates that reference *only* outer columns (`len(corCols) > 0 && len(cols) == 0`) — they are not decorrelation candidates at all — or otherwise evaluate them in the outer context; and reset the scratch slices before `continue`.

Related: #51116 (feature request), #63204 (master PR), #63541 (release-8.5 cherry-pick).

Contributor guide

Open the contributing guide

Research direction

Start with extractCorrelatedAggFuncs in pkg/planner/core/logical_plan_builder.go and review PlanBuilder.Init in pkg/planner/core/planbuilder.go for how noDecorrelate is set. Run the SQL reproduction with the session variable enabled and compare it with the disabled case. Done means outer aggregates retain the expected grouped result and correlated aggregate handling does not reuse stale column state.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.