Lightning error in calculating the number of duplicate records when tidb is down during import process
Open
affects-5.4
affects-6.0
affects-6.1
affects-6.2
affects-6.3
affects-6.4
affects-6.5
affects-6.6
affects-7.0
affects-7.1
affects-7.2
affects-7.3
affects-7.4
affects-7.5
affects-8.1
affects-8.5
component/lightning
impact/inconsistency
severity/major
type/bug
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
1. lightning import data, backend local, has many duplicate rows.
2. tidb oom during lightning deduplicate
### 2. What did you expect to see? (Required)
the number of duplicate rows is correct
### 3. What did you see instead (Required)
the number of duplicate rows is greater than real number
### 4. What is your TiDB version? (Required)
master
Contributor guide
Assessment
This issue has not been assessed yet.