lightning parallel import cause region count grows
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
use multiple lightning node to import
### 2. What did you expect to see? (Required)
the region count should be almost the same as there's only one lightning node
### 3. What did you see instead (Required)
the region count is #lightning times than the single lightning import
### 4. What is your TiDB version? (Required)
because lightning will call PD `SetRegionLabelRule`, and this API will split region on the request key range. And lightning uses actual data range as the key range. When there're multiple lightning node, each node has different key range. So each node will causes region split once
https://github.com/pingcap/tidb/blob/6c9ee36e8e7c86abbda374d841b019979ff3aaf4/pkg/lightning/backend/local/local.go#L1331-L1343
maybe consider to use the whole range of tableID, or tableID + record/ indexID in the request
Contributor guide
Assessment
This issue has not been assessed yet.