pingcap / pingcap/tidb

[lightning] TiDB Lightning can report success with a corrupted unique index when checksum and analyze are disabled

Open
#70,103 0 comments 0 reactions 0 assignees View on GitHub
component/lightning found-by-ai may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

**Environment**

Lightning a942e4684f; TiDB master 05b396fb66 retains the source root

**Steps**

Create a clustered-primary-key table with UNIQUE KEY uu(u). Import CSV rows (1,7) and (2,7) using backend=local, add-index-by-sql=false, conflict.strategy=replace, checkpoint.enable=false, post-restore.checksum=off, and post-restore.analyze=off. After Lightning exits successfully, compare IGNORE INDEX(uu), FORCE INDEX(uu), and ADMIN CHECK TABLE.

### 2. What did you expect to see? (Required)

The configured replace strategy must detect and resolve duplicate primary or unique keys independently of whether checksum and analyze are enabled. A successful import must leave record and index rowsets identical.

### 3. What did you see instead? (Required)

Lightning exits 0 and reports the whole procedure complete. IGNORE INDEX(uu) returns 1:7,2:7, FORCE INDEX(uu) returns only 1:7, and ADMIN CHECK TABLE fails with error 8223 for handle 2. Changing only checksum from off to required detects two conflicts, resolves them, and leaves one consistent row.

### 4. What is your TiDB version? (Required)

Lightning a942e4684f; TiDB master 05b396fb66 retains the source root

Likely root cause and fix direction

**Likely root cause**

postProcess has an early return when checksum and analyze are both off. Local duplicate collection and ResolveDuplicateRows are embedded later inside the checksum stage, so the early return treats two optional reporting stages as if no required conflict-resolution work remains.

**Fix direction**

Decouple duplicate detection and resolution from checksum and analyze. The early return may run only when the backend and conflict strategy prove no conflict work is required; alternatively reject this configuration until the stages are independent.

Contributor guide

Open the contributing guide

Research direction

Start in postProcess and trace the early return when checksum and analyze are both off, then follow the later local duplicate collection and ResolveDuplicateRows paths. Reproduce the stated local-backend configuration and verify that duplicate resolution still runs, the import reports the configured result, and record and index rowsets remain consistent under ADMIN CHECK TABLE.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.