Date Tiered Compaction
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
As far as I can tell, Lucene doesn’t currently have a date‑tiered merge policy. I’m considering adding one to better support temporal queries over recent time windows.
On our larger clusters we see frequent paging of index data in and out of the cache. While most users write in temporal order—and TieredMergePolicy often produces a layout similar to what a date‑tiered approach would—it can still create segments with overlapping time ranges. I’m proposing a time‑aware merge policy, similar in spirit to the “date‑tiered compaction” described here: https://engineering.atspotify.com/2014/12/date-tiered-compaction/
I’d appreciate guidance on two points:
1. 1. Is this feasible and something the community would want?
2. Is it worthwhile, or do current features/configurations already achieve something similar?
If this is something interesting to the community, I am happy to write up a design doc and work on this task.
Thanks
Contributor guide
Research direction
No source file or test is named. Start by reviewing Lucene’s existing TieredMergePolicy and related merge-policy configuration, then compare their behavior with the linked date-tiered compaction proposal. Done would require community agreement on feasibility and scope, followed by a design document rather than a narrowly defined code change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100