pingcap / pingcap/tidb

[ddl] Distributed ADD INDEX hangs in running/retry on persistent SetTSBeforeImportEngine context-deadline-exceeded errors

Open
#70,029 1 comment 0 reactions 1 assignee Claimed by @wjhuang2016 View on GitHub
affects-8.5 component/ddl component/DXF found-by-ai severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

> Evidence status: confirmed.

### 1. Minimal reproduce step (Required)

On testbed 8220955 with tidb_enable_dist_task=ON and tidb_ddl_enable_fast_reorg=ON, use a commit-matched failpoint owner and inject github.com/pingcap/tidb/pkg/ingestor/ingestctrl/mockAINativeSetTSBeforeImportEngineErr at SetTSBeforeImportEngine. One-shot control 1*return("context_deadline_exceeded") keeps job 4002 / task 300007 green. Persistent return("context_deadline_exceeded") keeps job 4007 and DXF task 300008 running for >90s; owner log records 247 repeated "meet retryable error" lines before clearing the failpoint, after which the same job syncs within about 2 seconds.

### 2. What did you expect to see? (Required)

A persistent retryable import/setup timeout should hit a bounded retry budget or escalate to a terminal failed/reverting state instead of leaving the distributed DDL running forever.

### 3. What did you see instead? (Required)

The DXF executor keeps context.DeadlineExceeded in the retry bucket, leaves the subtask running, and reruns it indefinitely; the job only finishes after external fault removal.

### 4. What is your TiDB version? (Required)

TiDB v9.0.0-beta.2.pre-1895-g5c9198e948 distributed ADD INDEX / DXF ingest path

Likely root cause and fix direction

DXF taskexecutor treats retryable subtask errors as stay-running with no per-subtask/per-DDL terminal retry budget, so a source-native retryable SetTSBeforeImportEngine timeout can bypass the lower ingest retry limit and become an unbounded rerun loop.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.