pingcap / pingcap/tidb

[ddl] MODIFY COLUMN rolls back on transient connection-family errors that sibling ADD INDEX retries through

Open
#70,027 1 comment 0 reactions 1 assignee Claimed by @wjhuang2016 View on GitHub
affects-7.5 affects-8.1 affects-8.5 component/ddl found-by-ai severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

> Evidence status: confirmed.

### 1. Minimal reproduce step (Required)

On testbed 8220955 (TiDB v9.0.0-beta.2.pre-1895-g5c9198e948), use the commit-matched failpoint owner fp-tidb built from 5c9198e948 and inject one classifier-bridge fault after runReorgJob returns and before the retry gate. Control: context_deadline_exceeded is GREEN on both ADD INDEX (job 1755) and MODIFY COLUMN (job 1758). RED/GREEN split: driver_bad_conn => ADD INDEX t_add_badconn job 1761 synced, MODIFY COLUMN t_mod_badconn job 1764 rollback done; net_conn_reset => ADD INDEX t_add_reset job 1767 synced, MODIFY COLUMN t_mod_reset job 1770 rollback done. Earlier bridge-proximal grpc unavailable also split cleanly: ADD INDEX job 1723 synced while MODIFY COLUMN job 1726 rollback done. Local end-to-end family probes in pkg/ddl/ai_native_reorg_grpc_probe_test.go extend the same root to mysql invalid connection, broken pipe, and connection refused.

### 2. What did you expect to see? (Required)

A one-shot transient foreign connection/grpc error during active MODIFY COLUMN reorg should stay on a retryable recovery path once the dependency is healthy again; it should not terminally roll back after a single hit when sibling ADD INDEX proves the same bridge/fault schedule is otherwise recoverable.

### 3. What did you see instead? (Required)

Live bridge-level evidence on testbed 8220955 shows MODIFY COLUMN jobs 1726, 1764, and 1770 end rollback done on grpc unavailable, driver: bad connection, and connection reset by peer, while same-environment sibling ADD INDEX jobs 1723, 1761, and 1767 all synced. The green control context_deadline_exceeded stayed synced on both sides. Local classifier probes also show raw transient connection/grpc errors stay retryable before DDL synthesis, while modify-column terminalizes them at the retry gate.

### 4. What is your TiDB version? (Required)

TiDB v9.0.0-beta.2.pre-1895-g5c9198e948 modify-column reorg path; local current-master family probes show the same bridge split

Likely root cause and fix direction

pkg/ddl/modify_column.go uses isRetryableModifyColumnReorgJobError(err, jobErrCnt) -> isRetryableError(err, false), while pkg/ddl/index.go uses isRetryableJobError(err, jobErrCnt) -> isRetryableError(err, true). Unknown foreign transient errors therefore keep retryability for index-family DDL but are fatalized for modify-column at the classifier bridge. The outer worker may still print a generic retry log after the inner modify-column path has already committed the job state to rollingback, so terminal job state is the reliable oracle.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.