apache / apache/lucene

Index level caching policy is thrashed by segment-specific query rewrites

Open
#14,986 7 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

1. Each IndexSearcher has its own UsageTrackingQueryCachingPolicy that is shared across all segments.
2. This caching policy uses a 256-length ring buffer to keep track of recently used queries.
3. A `TermInSetQuery` with `rewriteMethod = MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE` yields a RewritingWeight.
5. Getting a scorer from this RewritingWeight for a segment could involve rewriting to a BooleanQuery of multiple TermQuery with only the terms present in that particular segment - ref `org.apache.lucene.search.AbstractMultiTermQueryConstantScoreWrapper.RewritingWeight#scorerSupplier`
6. Thus a single TermInSetQuery will end up thrashing the ring buffer as multiple distinct `BooleanQuery`s from different segments.
7. This leads to a poor caching rate for indexes with a large number of segments.

We could verify this behavior with a new caching policy that delegates to `UsageTrackingQueryCachingPolicy` after logging the `onUse()` and `shouldCache()` calls.

Is there a good reason to not have this ring buffer tracking at a per segment level? That would fix this issue.

### Version and environment details

Lucene 9.12.1

Contributor guide

Open the contributing guide

Research direction

Start with UsageTrackingQueryCachingPolicy and AbstractMultiTermQueryConstantScoreWrapper.RewritingWeight#scorerSupplier. Reproduce the behavior using a delegating caching policy that logs onUse() and shouldCache() calls, then compare the current shared ring-buffer behavior with per-segment tracking. Done means the caching-rate impact and the suitability of a per-segment policy are established with supporting tests or measurements.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.