FairDistributionMergePolicy [LUCENE-10679]
Open
legacy-jira-priority:Major
type:enhancement
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
TieredMergePolicy and LogMergePolicy can define merge specifications which have a skew in the distribution of overall "work" (i.e. number of documents to process) amongst threads. This is especially true when the underlying segment distribution is highly skewed.
A more optimal distribution can be achieved by performing a variation of the integer partitioning algorithm. Initial tests show a more optimal distribution on a simulated set of skewed segment distributions.
---
Migrated from [LUCENE-10679](https://issues.apache.org/jira/browse/LUCENE-10679) by Atri Sharma (@atris)
Contributor guide
Assessment
This issue has not been assessed yet.