pingcap / pingcap/tidb

Support min number of region to split and scatter in tidb lightning

Open
#51,494 13 comments 1 reaction 0 assignees View on GitHub
component/lightning type/feature-request
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Feature Request

Currently in tidb lightning physical backend, it'll split and scatter regions before it imports the data. The number of regions it split depends on the size of the source data and the default region size of the tidb cluster. Can we add a parameter in lightning config to support the minimum number of regions it split? So that number of region it split = max(minimum per of region in the config, data size / region size)

**Is your feature request related to a problem? Please describe:**

we have a use case that user import a very small data(lower than 1G) and has a high QPS. After we use lightning to import the data, the table just has one region so all the traffic goes to this one region, which cause the tikv to be slow and also affect the traffic of other tables. This issue only resolved after the tikv split and scatter that region to other nodes. We think it's better to have the ability to let lightning to split more region for some tables.

**Describe the feature you'd like:**

**Describe alternatives you've considered:**

**Teachability, Documentation, Adoption, Migration Strategy:**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.