enhancement: add a bulk-load fast path for StorageEndpoint.LoadRegions
- Dominant language
- Go
- Stars
- 1.2k
- Forks
- 783
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 36
Description
## Enhancement Task
### Problem
Restoring PD with 100,015,214 regions took 18m14.087s in the process-only restart test. The hot path is `pkg/storage/endpoint/meta.go:174-221`, which does per-region `LoadRange -> proto.Unmarshal -> DecryptRegion -> core.NewRegionInfo -> CheckAndPutRegion` work, then walks overlap/subtree maintenance in `pkg/core/region.go:1095-1112` and `1257-1285`.
### Why this is slow
The restore path is effectively one region at a time. Even when the persisted data is already canonical and ordered, it still pays the full per-region parse / decrypt / tree update cost.
### Goal
Reduce 100M-scale restart recovery time without changing correctness.
### Possible directions
- Add a bulk-load fast path for ordered persisted regions.
- Reduce per-region allocations in `LoadRegions`.
- Avoid redundant overlap/subtree work when loading canonical data.
- Parallelize decode/decrypt before tree insertion if safe.
### Evidence
- 100M process-only restart: `load regions` completed in 18m14.087s.
- Code path: `pkg/storage/endpoint/meta.go:174-221`.
- Tree/subtree update path: `pkg/core/region.go:1095-1112`, `1257-1285`.
Contributor guide
Assessment
This issue has not been assessed yet.