ddl: old owner silently drops tidb_max_dist_task_nodes in mixed-version distributed backfill
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
This issue happens on the mixed-version split:
- new submitter: `v8.5.6`
- old DDL owner / dist-task scheduler: `v8.5.5`
A reliable runtime path is a partitioned add-index backfill task.
1. Prepare a mixed-version cluster with one old TiDB node (`v8.5.5`) and one new TiDB node (`v8.5.6`).
2. Make sure the new node is the DDL owner first, then verify the control path:
```sql
create database if not exists compat_66376;
use compat_66376;
create table t(
a bigint auto_random primary key,
b int
) partition by hash(a) partitions 20;
insert into t values
(1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7),(8,8),(9,9),(10,10),
(11,11),(12,12),(13,13),(14,14),(15,15),(16,16),(17,17),(18,18),(19,19),(20,20),
(21,21),(22,22),(23,23),(24,24),(25,25),(26,26),(27,27),(28,28),(29,29),(30,30);
split table t between (3) and (8646911284551352360) regions 50;
set @@session.tidb_max_dist_task_nodes=1;
alter table t add index idx_b(b);
```
On the new-owner path, the task row keeps `max_node_count=1` and all subtasks stay on one executor.
3. Now force the old node to become DDL owner by disabling DDL on the new node:
```sql
set @@global.tidb_enable_ddl = false;
```
4. From the new `v8.5.6` node, run the same task again with:
```sql
set @@session.tidb_max_dist_task_nodes=1;
alter table t add index idx_b(b);
```
5. Inspect:
- `mysql.tidb_global_task_history.max_node_count`
- `meta.job.reorg_meta.max_node_count`
- `mysql.tidb_background_subtask_history`
- owner log lines about `eligible instances`
### 2. What did you expect to see? (Required)
I expected the user-configured node cap to be honored even when the task is submitted from a new node while an old node is the DDL owner / dist-task scheduler.
Concretely:
- the persisted task row should keep `max_node_count=1`
- `meta.job.reorg_meta.max_node_count` should also keep `1`
- the task should stay on one executor
### 3. What did you see instead (Required)
On the mixed-version path `new submitter (v8.5.6) -> old owner/scheduler (v8.5.5)`:
- the persisted task row fell back to `max_node_count=0`
- `meta.job.reorg_meta.max_node_count` was omitted
- the task still fanned out across both TiDB executors (`10/10` subtasks)
- old owner log still reported `eligible instances ... num=2`
Control under the new-owner path behaved correctly:
- `max_node_count=1`
- all 20 subtasks stayed on one executor
So the new node-cap semantic is silently dropped when the old owner materializes the distributed task.
### 4. What is your TiDB version? (Required)
Mixed-version reproduction:
Old TiDB node:
```text
Release Version: v8.5.5
```
New TiDB node:
```text
Release Version: v8.5.6
```
### Additional context
This is a patch-compatibility / rolling-upgrade issue rather than a same-version functional bug.
The user-facing impact is that `tidb_max_dist_task_nodes` becomes effectively ignored in the `new submitter -> old owner/scheduler` path, so a task that was explicitly capped to one node may still fan out across multiple executors.
Reproduction was validated in a cloud mixed-version environment. A detailed local report artifact is available on my side if needed.
Contributor guide
Assessment
This issue has not been assessed yet.