checksum might mismatch with parallel import if some instances runs too fast
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
if some instance finished import right before we do checksum, the already imported data will be calculated twice, and cause checksum mismatch
https://github.com/pingcap/tidb/blob/851af3587ca670f129b456522c0fa53ae8929f2a/lightning/pkg/importer/meta_manager.go#L330-L332
it's very hard to reproduce this, it requires
- current instance should be very slow
- another instance runs very fast, and the data should be quite small
so just record this and mark as minor
### 2. What did you expect to see? (Required)
### 3. What did you see instead (Required)
### 4. What is your TiDB version? (Required)
Contributor guide
Research direction
Start by reading the checksum-related code at lightning/pkg/importer/meta_manager.go#L330-L332 and trace how parallel import completion is handled. Confirm the reported case where one instance finishes just before checksum calculation, then ensure already imported data is not counted twice and the checksum remains consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100