[lightning] TiDB Lightning can silently skip all data after the target table is recreated
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
> Evidence status: confirmed.
### 1. Minimal reproduce step (Required)
Configure classic tidb-lightning with a file checkpoint and keep-after-success="origin". Import a nonempty CSV into db.t and record the table ID. Drop and recreate db.t with the same schema, confirming that its table ID changed and it is empty. Run the same Lightning command again with the retained checkpoint, then inspect the process exit status and the current table rowset.
### 2. What did you expect to see? (Required)
Lightning must reject a checkpoint whose target-table generation does not match the current table, or reset that table checkpoint and import the current input from Loaded state.
### 3. What did you see instead? (Required)
On current master the first import produced table ID 5412 with rows 1 and 2. After recreation, table ID 5415 was empty. The second Lightning run exited 0 in under one second, but table ID 5415 remained empty; ADMIN CHECK TABLE was green because the empty table was internally consistent.
### 4. What is your TiDB version? (Required)
master 13282a8bd06b
Likely root cause and fix direction
Classic Lightning checkpoint tables are keyed by table name but are not bound to the target table generation. FileCheckpointDB.Initialize explicitly leaves hash validation as TODO and preserves old TableID/status/engines/chunks. The MySQL checkpoint path writes the constant CheckpointStatusLoaded value into its hash column. In the TiDB backend the persisted TableID is also 0, so a completed status and imported engines survive a real table recreation and dominate all current work and postprocess checks.
Contributor guide
Research direction
Start by reading FileCheckpointDB.Initialize and the MySQL checkpoint path described in the issue, then trace how the TiDB backend persists TableID, status, engines, and chunks. Reproduce the retained-checkpoint scenario with a recreated table. Done means Lightning rejects the stale checkpoint or resets it and imports the current input into the new table.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, mysql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100