Consider implementing "lookback" merge policy [LUCENE-5134]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
In http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html Mike mentioned lookahead as something that could possibly yield more
optimal merges.
But what about lookback?
What if some sort of stats were kept about about which segments were
picked for merges? With some sort of stats in hand, could one look
back and, knowing what happened after those merges, evaluate if more
optimal merge choices could have been made and then use that "next
time"?
See http://search-lucene.com/m/D7ypz1gT2H91
---
Migrated from [LUCENE-5134](https://issues.apache.org/jira/browse/LUCENE-5134) by Otis Gospodnetic (@otisg)
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by reading the linked merge discussion and locating Lucene’s segment merge policy implementation; then define the required statistics, feedback loop, and evaluation criteria before assessing feasibility. Done would require an agreed design and tests demonstrating whether lookback produces better merge choices.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100