Optimize the frequency of BatchLoadRegionsWithKeyRange to avoid large performance impaction on pd
Open
Nobody has claimed this yet.
type/enhancement
- Dominant language
- Go
- Stars
- 56
- Forks
- 63
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
TiCDC use BatchLoadRegionsWithKeyRange to convert range requests to region requests, we already have a limit on the concurrency of calling BatchLoadRegionsWithKeyRange. But in scenario with many region error, the calling of BatchLoadRegionsWithKeyRange seems to influence the performance of pd. We need test it and seems whether there is any optimization to improve the scenario.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing TiCDC's use of BatchLoadRegionsWithKeyRange and reproduce the many-region-error scenario described in the issue. Measure its effect on PD, then determine and validate an optimization that reduces the impact without breaking range-to-region conversion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100