lightning: registerTaskToPD causes context deadline exceeded under high parallelism
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report / Feature Request
### Problem Summary
When many TiDB Lightning jobs run concurrently against the same TiDB/PD cluster, the `registerTaskToPD` RPC frequently fails with `context deadline exceeded`. PD serialises task registrations via etcd leases, so at high concurrency (e.g. 1,000+ concurrent jobs) the registration queue drains slowly and individual jobs time out before they can register.
### Reproduction
Run 100+ Lightning jobs simultaneously against the same cluster with `pause-pd-scheduler-scope = "table"` (default) or `"global"`. Monitor logs for:
```
context deadline exceeded
```
during the PD task registration phase at import start.
### Expected Behavior
Lightning should provide a way to skip PD scheduler registration entirely when the operator knows the cluster is under controlled high-parallelism ingestion and wants to manage scheduling externally.
### Suggested Solution
Add `pause-pd-scheduler-scope = "off"` as a valid configuration value that skips:
- `registerTaskToPD` (etcd registration under `/tidb/brie/import/lightning/...`)
- `CheckAndPausePdSchedulers` (global scheduler removal)
- `CheckAndFinishRestore` (scheduler restore lifecycle)
The trade-off is that BR log backup cannot detect active Lightning runs. This is acceptable when the operator ensures BR point-in-time restore is not run concurrently with the ETL pipeline.
### Environment
- TiDB version: v8.x+
- Deployment: Many concurrent Lightning instances (e.g. partition-level ETL pipelines)
Contributor guide
Assessment
This issue has not been assessed yet.