pingcap / pingcap/tidb

checksum might mismatch with parallel import if some instances runs too fast

Open
#57,399 0 comments 0 reactions 0 assignees View on GitHub
component/lightning severity/minor type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

if some instance finished import right before we do checksum, the already imported data will be calculated twice, and cause checksum mismatch
https://github.com/pingcap/tidb/blob/851af3587ca670f129b456522c0fa53ae8929f2a/lightning/pkg/importer/meta_manager.go#L330-L332
it's very hard to reproduce this, it requires
- current instance should be very slow
- another instance runs very fast, and the data should be quite small

so just record this and mark as minor

### 2. What did you expect to see? (Required)

### 3. What did you see instead (Required)

### 4. What is your TiDB version? (Required)

Contributor guide

Open the contributing guide

Research direction

Start by reading the checksum-related code at lightning/pkg/importer/meta_manager.go#L330-L332 and trace how parallel import completion is handled. Confirm the reported case where one instance finishes just before checksum calculation, then ensure already imported data is not counted twice and the checksum remains consistent.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.