pingcap / pingcap/tidb

[lightning] TiDB Lightning can silently skip all data after the target table is recreated

Open
#70,041 0 comments 0 reactions 0 assignees View on GitHub
component/lightning found-by-ai may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

> Evidence status: confirmed.

### 1. Minimal reproduce step (Required)

Configure classic tidb-lightning with a file checkpoint and keep-after-success="origin". Import a nonempty CSV into db.t and record the table ID. Drop and recreate db.t with the same schema, confirming that its table ID changed and it is empty. Run the same Lightning command again with the retained checkpoint, then inspect the process exit status and the current table rowset.

### 2. What did you expect to see? (Required)

Lightning must reject a checkpoint whose target-table generation does not match the current table, or reset that table checkpoint and import the current input from Loaded state.

### 3. What did you see instead? (Required)

On current master the first import produced table ID 5412 with rows 1 and 2. After recreation, table ID 5415 was empty. The second Lightning run exited 0 in under one second, but table ID 5415 remained empty; ADMIN CHECK TABLE was green because the empty table was internally consistent.

### 4. What is your TiDB version? (Required)

master 13282a8bd06b

Likely root cause and fix direction

Classic Lightning checkpoint tables are keyed by table name but are not bound to the target table generation. FileCheckpointDB.Initialize explicitly leaves hash validation as TODO and preserves old TableID/status/engines/chunks. The MySQL checkpoint path writes the constant CheckpointStatusLoaded value into its hash column. In the TiDB backend the persisted TableID is also 0, so a completed status and imported engines survive a real table recreation and dominate all current work and postprocess checks.

Contributor guide

Open the contributing guide

Research direction

Start by reading FileCheckpointDB.Initialize and the MySQL checkpoint path described in the issue, then trace how the TiDB backend persists TableID, status, engines, and chunks. Reproduce the retained-checkpoint scenario with a recreated table. Done means Lightning rejects the stale checkpoint or resets it and imports the current input into the new table.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, mysql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.