pingcap / pingcap/tidb

Console output and log message not accurate when import data with dup records using lightning tidb backend.

Open
#54,081 2 comments 0 reactions 0 assignees View on GitHub
component/lightning severity/minor type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)
1. Lightning import some data with few duplicate rows using tidb end, conflict.strategy = 'replace';
```
# cat /tmp/tidb-lightning.toml
[lightning]
level = 'debug'
[mydumper]
filter = ['coms.PK_*']
[conflict]
strategy = 'replace'
```
2, Lighting import finishes

### 2. What did you expect to see? (Required)
Lightning import should succeed as there are only 5 duplicate rows.
And the info in lightning console output and log should be accurate.

### 3. What did you see instead (Required)
Lightning logs says that "Detect 5 conflict errors in total, please refer to table `lightning_task_info`.`conflict_view` for more details"
However, the conflict rows will not be recorded for strategy = "replace" in TiDB backend per https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-configuration, also validated that there is no records in the tables actually.
```
[2024/06/17 15:04:36.751 +00:00] [INFO] [import.go:578] ["the whole procedure completed"] [takeTime=1h6m44.025684115s] []
[2024/06/17 15:04:36.751 +00:00] [WARN] [errormanager.go:1089] ["Detect 5 conflict errors in total, please refer to table `lightning_task_info`.`conflict_view` for more details"]
[2024/06/17 15:04:36.751 +00:00] [INFO] [main.go:110] ["tidb lightning exit"] [finished=true]
```
![image](https://github.com/pingcap/tidb/assets/7403864/f0e3bf5e-2149-42a2-9fa8-d4224768e247)
![image](https://github.com/pingcap/tidb/assets/7403864/763b1189-6a7f-4f6b-8ac6-d55c7a33a12d)

### 4. What is your TiDB version? (Required)
Release Version: v8.2.0-alpha
Git Commit Hash: 26d10965804dcfc24c69720cbef7b8ecfb025068
Git Branch: heads/refs/tags/v8.2.0-alpha
Go Version: go1.21.10
UTC Build Time: 2024-06-17 11:41:27
Race Enabled: false

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.