[import] Classic IMPORT INTO can finish after a concurrent ADD UNIQUE INDEX and publish a corrupt unique index
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
# [import] Classic IMPORT INTO can finish after a concurrent ADD UNIQUE INDEX and publish a corrupt unique index
## Bug Report
### 1. Minimal reproduce step (Required)
This issue is about the Classic `IMPORT INTO` path on an empty target table.
The core problem is that Classic import captures target schema `S0` during planning, performs unguarded preparation work, and only later acquires `TableModeImport`. If an ordinary `ADD UNIQUE INDEX` publishes schema `S1` between those two moments, workers still encode with stale `S0`, while the finished job resets TableMode and reports success against the current `S1` table.
The production-shaped reproducer is:
1. Start `IMPORT INTO` on an empty table.
2. Let file discovery/prechecks keep the job in preparation long enough.
3. In another session, run `ALTER TABLE ... ADD UNIQUE INDEX`.
4. Wait for both operations to report success.
Maintainer witness on current master:
```bash
go test -tags=intest ./tests/realtikvtest/importintotest \
-run '^TestImportInto/TestAINativeImportIntoAddIndexDuringNaturalFileDiscovery$' \
-count=3 -timeout 15m
```
Environment from the reproduced RED:
- TiDB: `05b396fb6636f73b3bc06b09107cf43f2c725c35`
- one in-process TiDB, one PD, one real TiKV
- Classic kernel
- MDL enabled
- default strict SQL mode
- required checksum enabled
The natural schedule used:
- source directory with `60,000` unrelated readable entries plus one matching CSV
- session A starts `IMPORT INTO`
- session B runs `ALTER TABLE t ADD UNIQUE INDEX kv(v)` after `250ms`
There is also an exact deterministic witness that changes only the schedule control:
```bash
make failpoint-enable
go test -tags=intest ./tests/realtikvtest/importintotest \
-run '^TestImportInto/TestAINativeImportIntoAddIndexAfterTargetResolution$' \
-count=1 -timeout 15m
make failpoint-disable
```
That callback fires at `importer.NewImportPlan` and schedules only an ordinary `ADD UNIQUE INDEX`. No component/data failure is injected.
### 2. What did you expect to see? (Required)
Once `IMPORT INTO` later acquires `TableModeImport`, it should only proceed if the captured schema proof is still current.
If target schema changed from `S0` to `S1` in the gap, import must fail or restart planning. It must not publish success while the current unique index is missing all imported rows.
### 3. What did you see instead (Required)
The natural real-TiKV RED is:
- `ADD UNIQUE INDEX`: success
- `IMPORT INTO`: finished
- record scan: `1:101, 2:102, 3:103`
- `FORCE INDEX(kv)`: empty
- required checksum: pass
- `INSERT (4,101)`: success
- `ADMIN CHECK TABLE`: `ERROR 8223`
So the table contains the imported rows, but the current unique index contains none of them, and later duplicate business keys are accepted.
The exact callback RED reproduces the same terminal with no injected product/data error:
- `ADD UNIQUE INDEX`: success
- `IMPORT INTO`: finished
- imported records: `3`
- current unique-index rows for imported data: `0`
- duplicate insert on the new unique key: success
- `ADMIN CHECK TABLE`: `ERROR 8223`
The matched GREEN changes only the schedule:
- let `ADD UNIQUE INDEX` finish before `IMPORT INTO` planning
- import still finishes
- record scan: `3`
- unique-index scan: `3`
- duplicate insert: `ERROR 1062`
- `ADMIN CHECK TABLE`: pass
This isolates the bug to the stale-schema gap between proof capture and later TableMode claim.
#### Likely root cause
Current Classic import owners line up like this:
- proof owner: `Plan.TableInfo` copied from `tbl.Meta()` during `NewImportPlan`
- unguarded gap: object listing, format detection, size estimation, active-job/empty-table precheck, TiKV config initialization
- guard owner: `AlterTableMode(..., TableModeImport)` during task submission
- irreversible consumers: worker encoders, Lightning engines, TiKV SST ingest
The guard blocks only future changes. It does not atomically compare the expected schema generation captured earlier.
So if `ADD UNIQUE INDEX` publishes `S1` after planning captured `S0` but before `TableModeImport` is acquired, the import still encodes with `S0`, ingests no KVs for the `S1` unique index, then resets TableMode and reports a finished job against the current schema.
### 4. What is your TiDB version? (Required)
```text
TiDB: 05b396fb6636f73b3bc06b09107cf43f2c725c35
Store: tikv
Kernel: Classic IMPORT INTO
```
The reproduced REDs used one TiDB, one PD, one real TiKV, MDL enabled, default strict SQL mode, and required checksum.
### Impact
This is reachable through ordinary operations:
1. a bulk loader starts Classic `IMPORT INTO` on an empty table,
2. an independent schema migration adds a business-key unique index,
3. normal file/object discovery latency lets the DDL finish during import preparation,
4. both operations report success,
5. the current unique index is empty for the imported rows and later duplicate business keys are accepted.
No failpoint, retry, cancellation, multiple TiDB nodes, disabled MDL, nondefault SQL mode, or injected component fault is required for the natural RED.
Contributor guide
Research direction
Run the two named import integration tests, including the deterministic failpoint case, to reproduce the stale-schema gap. Read importer.NewImportPlan, Plan.TableInfo, and the later AlterTableMode(..., TableModeImport) path to trace schema proof capture and claiming. Done means a concurrent ADD UNIQUE INDEX cannot produce a successful import with missing index entries; the regression tests should pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100