apache / apache/lucene

Cache costly subqueries asynchronously [LUCENE-8213]

Open
#9,260 46 comments 0 reactions 0 assignees View on GitHub
affects-version:7.2.1 legacy-jira-label:performance legacy-jira-priority:Minor module:core/query/scoring type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

IndexOrDocValuesQuery allows to combine costly range queries with a selective lead iterator in an optimized way. However, the range query at some point gets cached by a querying thread in LRUQueryCache, which negates the optimization of IndexOrDocValuesQuery for that specific query.

It would be nice to see an asynchronous caching implementation in such cases, so that queries involving IndexOrDocValuesQuery would have consistent performance characteristics.

---
Migrated from [LUCENE-8213](https://issues.apache.org/jira/browse/LUCENE-8213) by Amir Hadadi, updated Nov 28 2019
Attachments: [0001-Reproduce-across-segment-caching-of-same-query.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8213/0001-Reproduce-across-segment-caching-of-same-query.patch), [thetaphi_Lucene-Solr-master-Linux_24839.log.txt](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8213/thetaphi_Lucene-Solr-master-Linux_24839.log.txt)
Linked issues:
- #10045

Contributor guide

Open the contributing guide

Research direction

Start with IndexOrDocValuesQuery and LRUQueryCache, then review the linked issue and the attached reproduction patch for the cross-segment caching behavior. The work is done when costly subqueries can be cached asynchronously without undermining IndexOrDocValuesQuery's optimized lead-iterator performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.