pingcap / pingcap/tidb

[ddl] Fast reorg ADD INDEX rolls back on transient PD TSO stream retry timeout instead of retrying

Open
#70,025 2 comments 0 reactions 0 assignees View on GitHub
affects-7.5 affects-8.1 affects-8.5 component/ddl found-by-ai severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

> Evidence status: confirmed.

### 1. Minimal reproduce step (Required)

On testbed 8220955 (TiDB v9.0.0-beta.2.pre-1895-g5c9198e948), run a large split-table ADD INDEX with tidb_enable_dist_task=OFF and tidb_ddl_enable_fast_reorg=ON, then bounce PD twice during write reorganization. The user statement returns ERROR 1105 create TSO stream failed, retry timeout. mysql.tidb_ddl_history for jobs 1192 and 1204 shows err.rfccode=PD:client:ErrClientCreateTSOStream, err_count=1, is_fast_reorg=true, is_dist_reorg=false, state=3. Control: the same schedule with fast_reorg=OFF succeeds on the txn path.

### 2. What did you expect to see? (Required)

A transient PD TSO stream creation failure during active fast-reorg ADD INDEX should stay on a retryable recovery path once PD is healthy again; it should not terminally roll back after a single retry-family error.

### 3. What did you see instead? (Required)

Live jobs 1192 and 1204 both ended rollback done after one hit of PD:client:ErrClientCreateTSOStream. The user-visible error was create TSO stream failed, retry timeout. A local classifier probe in pkg/ddl/ai_native_retry_probe_test.go shows isRetryableError(raw,true)=false for the raw PD normalized error.

### 4. What is your TiDB version? (Required)

TiDB v9.0.0-beta.2.pre-1895-g5c9198e948 fast reorg ingest path

Likely root cause and fix direction

DDL reorg retry classification routes the PD normalized error through the *terror.Error branch. terror.ToSQLError sees RFC class PD as unknown, falls back to a generic MySQL code, and isRetryableError therefore misses ReorgRetryableErrCodes / retryUnknown. The transient TSO retry-timeout error is treated as fatal instead of retryable.

Contributor guide

Open the contributing guide

Research direction

Start with pkg/ddl/ai_native_retry_probe_test.go and the isRetryableError entry point; trace how the *terror.Error branch and terror.ToSQLError handle the normalized PD error. Done means the transient ErrClientCreateTSOStream retry-timeout is classified as retryable and a regression test passes without terminal rollback.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.