pingcap / pingcap/tidb

Undersize regions after lightning import is finished

Open
#46,204 1 comment 0 reactions 0 assignees View on GitHub
affects-6.1 affects-6.5 affects-8.5 component/lightning severity/major type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report
If ingestion fails partially lightning again try to split the regions into unfinished ranges https://github.com/pingcap/tidb/blob/617497366ac4f009880196b27fd5f4b7923d118e/br/pkg/lightning/backend/local/local.go#L1099

It lead to lot of undersize regions in tikv. Split_and_scatter is not required once it is finished successfully. PD is not going to merge the regions as scheduler is paused.

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)
1. Run lightning in physical parallel import mode for 100 G of data.
2. Do region stats and you will see lot of undersize regions.
3.

### 2. What did you expect to see? (Required)
Minimal undersize regions.
### 3. What did you see instead (Required)
Lot of undersize regions
### 4. What is your TiDB version? (Required)
6.5.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.