pingcap / pingcap/tidb

MPP shared CTE with UNION ALL producer can fail planning on partially overlapping TiFlash workers

Open
#69,646 0 comments 0 reactions 0 assignees View on GitHub
component/tiflash may-affects-8.5 severity/major sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Related to #68837 and #68848.

After #68848, MPP shared CTE uses TiFlash-address-local `CteSinkNum` and `CteSourceNum`, which fixes the previous global-count hang risk. However, a producer-side `UNION ALL` can still produce a valid SQL shape whose split `CTESink` fragments have partially overlapping TiFlash worker sets. In that case, the current TiPB fields cannot represent the required per-address local counts, so TiDB returns a planner error without falling back to another valid plan.

### 1. Minimal reproduce step (Required)

Use a TiFlash MPP setup with at least three TiFlash workers and enable MPP shared CTE execution:

```sql
set @@tidb_enforce_mpp = 'on';
set @@tidb_opt_enable_mpp_shared_cte_execution = 'on';

create table t_ab(a int, b int);
create table t_bc(a int, b int);
alter table t_ab set tiflash replica 1;
alter table t_bc set tiflash replica 1;
```

Arrange the scan tasks so the two CTE producer branches are scheduled to partially overlapping TiFlash workers:

```text
t_ab scan tasks: [tiflash0, tiflash1]
t_bc scan tasks: [tiflash1, tiflash2]
```

Then run:

```sql
with c as (
select a, b from t_ab
union all
select a, b from t_bc
)
select x.a, y.b
from c as x
join c as y on x.a = y.a;
```

#### Expected fragment layout and local counts

The corresponding fragment layout is:

```text
CTE id: CTE_0

F0: CTESink(CTE_0) <- TableScan(t_ab)
self tasks: [tiflash0, tiflash1]

F1: CTESink(CTE_0) <- TableScan(t_bc)
self tasks: [tiflash1, tiflash2]

F2: ExchangeSender(Broadcast) <- CTESource(CTE_0 as x)
self tasks: [tiflash0, tiflash1, tiflash2]

F3: HashJoin <- ExchangeReceiver(F2) + CTESource(CTE_0 as y)
self tasks: [tiflash0, tiflash1, tiflash2]
```

The local counts are:

```text
sinkCounts:
tiflash0 = 1
tiflash1 = 2
tiflash2 = 1

sourceCounts:
tiflash0 = 2
tiflash1 = 2
tiflash2 = 2
```

When filling `F0`, one `CTESink` plan node spans `[tiflash0, tiflash1]`, but it would need `CteSinkNum=1` on `tiflash0` and `CteSinkNum=2` on `tiflash1`. A single TiPB `CTESink` node only has one scalar `CteSinkNum`, so this layout cannot be represented.

### 2. What did you expect to see? (Required)

TiDB should execute the valid SQL correctly, for example by falling back from MPP shared CTE to a non-shared CTE plan, a non-MPP CTE plan, or another representable MPP layout.

### 3. What did you see instead (Required)

TiDB returns a planner-stage error similar to:

```text
MPP shared CTE 0 has different local sink counts in one fragment
```

This is fail-fast and does not produce wrong results, but it makes a valid SQL query fail when MPP shared CTE is selected.

### 4. What is your TiDB version? (Required)

master after #68848.

Local analysis was based on TiDB commit `d5729fb1bc47e2a1de9b6c0f91d052fb40907b5f` plus the MPP shared CTE planner/TiFlash runtime contract. `SELECT tidb_version()` output is not attached because this report is based on planner fragment generation and not an end-to-end cluster run.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the SQL with at least three TiFlash workers and tracing MPP shared CTE planner fragment generation and the TiPB CTESink fields. Inspect how per-address sink and source counts are represented for the shown fragment layout. Done means the valid query no longer fails at planning and uses a representable shared-CTE layout or a valid fallback plan.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.