[ddl] Failed server-info restart can let MDL DDL corrupt a secondary index
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
> Evidence status: confirmed.
### 1. Minimal reproduce step (Required)
On TiDB A, begin a pessimistic transaction and insert one row. Let only A's 90-second server-info etcd session end while its schema-sync session and SQL connection remain live. A creates a replacement session, but all five StoreServerInfo retries fail during a short etcd recovery flap; the replacement lease then remains healthy. Before that lease ends, TiDB B runs ADD INDEX and A commits the old transaction. MDL remains at its default ON value. The real-TiKV reproducer closes exactly the server-info session, fails exactly the first restart publication, then lets etcd recover.
### 2. What did you expect to see? (Required)
A failed registration must retain a retry trigger or fail closed. DDL must include every live old-schema TiDB in its MDL wait set, or the old transaction must fail schema validation.
### 3. What did you see instead? (Required)
Restart logged mock store server info error and never retried because the newly assigned session remained live. ADD INDEX succeeded, COMMIT succeeded, a table scan returned (1,10), a forced idx_v scan returned no rows, and ADMIN CHECK TABLE returned 8223.
### 4. What is your TiDB version? (Required)
TiDB 2964713e / real TiKV nightly; classic defaults; MDL ON; server-info TTL 90s
Likely root cause and fix direction
NewSessionAndStoreServerInfo assigns s.session before StoreServerInfo succeeds. If StoreServerInfo exhausts its retries, ServerInfoSyncLoop returns to select on the live replacement session instead of the completed old session, so no retry republishes /tidb/server/info. MDL DDL builds its wait set from that key. Meanwhile MDL transactions set needCheckSchemaByDelta=false, so the old transaction trusts the missing DDL wait and commits against the new schema.
Contributor guide
Research direction
Start by tracing NewSessionAndStoreServerInfo and ServerInfoSyncLoop, then inspect how MDL DDL builds its wait set from /tidb/server/info. Reproduce the failed server-info restart with the real-TiKV setup described, and verify that registration retries or fails closed so ADD INDEX cannot commit alongside the old transaction; confirm with the table scan, idx_v scan, and ADMIN CHECK TABLE.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100