pingcap / pingcap/tidb

[ddl] Failed server-info restart can let MDL DDL corrupt a secondary index

Open
#70,045 1 comment 0 reactions 0 assignees View on GitHub
affects-7.5 affects-8.1 affects-8.5 component/ddl found-by-ai severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

> Evidence status: confirmed.

### 1. Minimal reproduce step (Required)

On TiDB A, begin a pessimistic transaction and insert one row. Let only A's 90-second server-info etcd session end while its schema-sync session and SQL connection remain live. A creates a replacement session, but all five StoreServerInfo retries fail during a short etcd recovery flap; the replacement lease then remains healthy. Before that lease ends, TiDB B runs ADD INDEX and A commits the old transaction. MDL remains at its default ON value. The real-TiKV reproducer closes exactly the server-info session, fails exactly the first restart publication, then lets etcd recover.

### 2. What did you expect to see? (Required)

A failed registration must retain a retry trigger or fail closed. DDL must include every live old-schema TiDB in its MDL wait set, or the old transaction must fail schema validation.

### 3. What did you see instead? (Required)

Restart logged mock store server info error and never retried because the newly assigned session remained live. ADD INDEX succeeded, COMMIT succeeded, a table scan returned (1,10), a forced idx_v scan returned no rows, and ADMIN CHECK TABLE returned 8223.

### 4. What is your TiDB version? (Required)

TiDB 2964713e / real TiKV nightly; classic defaults; MDL ON; server-info TTL 90s

Likely root cause and fix direction

NewSessionAndStoreServerInfo assigns s.session before StoreServerInfo succeeds. If StoreServerInfo exhausts its retries, ServerInfoSyncLoop returns to select on the live replacement session instead of the completed old session, so no retry republishes /tidb/server/info. MDL DDL builds its wait set from that key. Meanwhile MDL transactions set needCheckSchemaByDelta=false, so the old transaction trusts the missing DDL wait and commits against the new schema.

Contributor guide

Open the contributing guide

Research direction

Start by tracing NewSessionAndStoreServerInfo and ServerInfoSyncLoop, then inspect how MDL DDL builds its wait set from /tidb/server/info. Reproduce the failed server-info restart with the real-TiKV setup described, and verify that registration retries or fails closed so ADD INDEX cannot commit alongside the old transaction; confirm with the table scan, idx_v scan, and ADMIN CHECK TABLE.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.