Reduce the number of regions getting paused scheduling during lightning physical backend import
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Feature Request
when using lightning physical backend to import data, it'll paused the scheduling of all regions of the target table. For example, if a table has 1024 regions, the scheduling of all of them will be paused until the import is finished, even though there're only 4 threads in lightning are importing at the same time.
What I think may be improved is that, the number of regions that should be paused scheduling should equal to the number of import threads, which is 4 in the above example, instead of all 1024 regions. That may require to move the call of `PauseSchedulersByKeyRange` and `SplitAndScatterRegionInBatches` to the worker inside `writeAndIngestByRanges`.
We want this feature to be on lightning 6.5. Please let me know if you have any concern about this.
**Is your feature request related to a problem? Please describe:**
The reason we want to reduce the number of regions getting paused scheduling is that, our TiDB cluster is running on the k8s cluster. In the k8s cluster we have a process of node drainer which will periodically drainer a k8s node and then replace it with a new node. During the draining, it'll try to kill the TiKV pod and schedule it to another node. But if some regions on the TiKV are paused scheduling , TiKV can not evict leader and block the pod to be rescheduled to another node. If the lightning takes too long to import data, this whole process will be blocked for a long time. So we hope to reduce the number of regions getting paused scheduling in order to reduce the time of node drainer getting blocked.
**Describe the feature you'd like:**
**Describe alternatives you've considered:**
**Teachability, Documentation, Adoption, Migration Strategy:**
Contributor guide
Assessment
This issue has not been assessed yet.