pingcap / pingcap/tidb

[import] Concurrent same-table IMPORT INTO jobs can leave persistent row/index corruption

Open
#70,090 1 comment 0 reactions 0 assignees View on GitHub
component/import found-by-ai may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/moderate type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

**Environment**

TiDB master 231dad5225; Classic nightly ed2376acc6; NextGen CSE ce46fc5067; real PD/TiKV; MDL enabled

**Steps**

Create t(v VARCHAR(64) NOT NULL, UNIQUE KEY uk(v)) with no primary key and two disjoint CSV files. From two sessions submit IMPORT INTO t FROM each file WITH DETACHED at nearly the same time. On Classic, local server paths are sufficient and all import settings can remain default. Wait for both jobs, compare SELECT COUNT(*) FROM t USE INDEX() with FORCE INDEX(uk), probe one value from each file, and run ADMIN CHECK TABLE. Repeat with one job or submit the second after the first is running as controls.

### 2. What did you expect to see? (Required)

Admission and owner publication atomically allow at most one active import for a target table. Table mode must identify and fence its owning job. A second request fails before irreversible ingest, and no terminal combination leaves record/index disagreement.

### 3. What did you see instead? (Required)

Classic current-nightly natural runs admitted both jobs within 11-130 microseconds. With 1,000,000 rows per disjoint input and default settings, both crossed ingest: one job finished, one failed checksum, the record/index counts were 1,000,000/2,000,000, a losing-key lookup returned the winner row, and ADMIN CHECK returned 8223. A 100,000-row default run produced the same 100,000/200,000 RED. NextGen independently reproduced the same invariant failure 3/3.

### 4. What is your TiDB version? (Required)

TiDB master 231dad5225; Classic nightly ed2376acc6; NextGen CSE ce46fc5067; real PD/TiKV; MDL enabled

Likely root cause and fix direction

**Likely root cause**

GetActiveJobCnt reads pending/running jobs before CreateJob publishes the new owner, with no target-unique atomic claim. NextGen skips Classic TableModeImport. Classic sets TableModeImport, but TableMode has no owner identity and CanTransitionTo explicitly permits Import-to-Import, so two submissions that plan concurrently can both publish jobs. Both plans allocate hidden handles from the same empty-table state; record keys collide while distinct unique-index keys survive. Checksum runs after durable SST ingest and cannot roll it back.

**Fix direction**

Atomically acquire a durable per-target active-owner lease with job creation, keyed by keyspace and stable table identity. Store the owner job ID or fencing token in table mode or a unique metadata row, reject same-mode transitions from a different owner, retain the claim through ingest and post-processing, and release it only after all sibling owners terminate.

Contributor guide

Open the contributing guide

Research direction

Reproduce the concurrent IMPORT INTO scenario with two sessions and compare record/index counts plus ADMIN CHECK TABLE. Start by tracing GetActiveJobCnt, CreateJob, TableModeImport, and CanTransitionTo to understand admission and ownership publication. Done means one active owner is admitted per target table, the competing job is rejected before irreversible ingest, and no terminal state leaves record/index disagreement.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.