Index level caching policy is thrashed by segment-specific query rewrites
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
1. Each IndexSearcher has its own UsageTrackingQueryCachingPolicy that is shared across all segments.
2. This caching policy uses a 256-length ring buffer to keep track of recently used queries.
3. A `TermInSetQuery` with `rewriteMethod = MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE` yields a RewritingWeight.
5. Getting a scorer from this RewritingWeight for a segment could involve rewriting to a BooleanQuery of multiple TermQuery with only the terms present in that particular segment - ref `org.apache.lucene.search.AbstractMultiTermQueryConstantScoreWrapper.RewritingWeight#scorerSupplier`
6. Thus a single TermInSetQuery will end up thrashing the ring buffer as multiple distinct `BooleanQuery`s from different segments.
7. This leads to a poor caching rate for indexes with a large number of segments.
We could verify this behavior with a new caching policy that delegates to `UsageTrackingQueryCachingPolicy` after logging the `onUse()` and `shouldCache()` calls.
Is there a good reason to not have this ring buffer tracking at a per segment level? That would fix this issue.
### Version and environment details
Lucene 9.12.1
Contributor guide
Research direction
Start with UsageTrackingQueryCachingPolicy and AbstractMultiTermQueryConstantScoreWrapper.RewritingWeight#scorerSupplier. Reproduce the behavior using a delegating caching policy that logs onUse() and shouldCache() calls, then compare the current shared ring-buffer behavior with per-segment tracking. Done means the caching-rate impact and the suitability of a per-segment policy are established with supporting tests or measurements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100