Segment merge is too slow to catch up segment split
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Enhancement
Currently, we always split a new segment to hold the dtfile of a region snapshot, in the case of adding new TiFlash instances. The number of segments is equal to the regions if the segment merge does not work. But since there is only 1 background thread that runs segment merge, the speed of segment merge fails to catch up with segment split, especially after the configuration of store limit has been increased.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Start by locating the TiFlash implementation and tests for segment splitting, segment merging, and the single background merge thread; use the reported workload and screenshots to understand the lag. Done should mean segment merge can catch up with segment split under the increased store-limit configuration, with tests or benchmarks showing the improvement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100