statistics: add concurrency for async merge global stats to improve performance
Open
type/enhancement
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
### Background
In the new TiDB version, `async merge global stats` feature is supported to reduce memory usage and merge stats time.
Async merge global stats is really important for our business clusters, because more than 100+ partitioned tables exist and each partition is partitioned by day range. The largest number of partition now is about 2200+.
But in my sysbench test, merging 4096 partition stats takes about **8 mins** and it brings some risks for online.
### Optimization
parallel merge fmsketch/cmsketch/histogramAndTopN, so that the performance of merging 4096 partition stats improves about 30%.
Contributor guide
Assessment
This issue has not been assessed yet.