lightning: add target-partition config to ingest directly into a named partition
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Summary
TiDB Lightning currently writes all SSTs into the full table key range when loading into a partitioned table. This makes it impossible to run concurrent Lightning jobs targeting different partitions of the same table — they conflict during the local sort and ingest phase.
## Motivation
For customers with large partitioned tables (e.g. LIST COLUMNS partitioned by customer), we need to run many independent Lightning jobs in parallel, each loading data for one partition. Today this requires a temp table + EXCHANGE PARTITION dance which is slow and error-prone.
## Proposed Change
Add a `target-partition` config option to `[mydumper]`:
```toml
[mydumper]
target-partition = "p_acme_2026_04_19"
```
When set, Lightning scopes all operations to the named partition:
- **Empty pre-check**: checks only the target partition
- **Checksum**: runs `ADMIN CHECKSUM TABLE t PARTITION (p)` instead of full-table
- **ANALYZE**: runs `ANALYZE TABLE t PARTITION (p)` instead of full-table
This also requires extending the `ADMIN CHECKSUM TABLE` grammar to accept a `PARTITION (...)` clause.
## Related PR
https://github.com/pingcap/tidb/pull/67892
Contributor guide
Assessment
This issue has not been assessed yet.