apache / apache/lucene

IndexSearcher#search(Query,int) should operate on a shared priority queue when configured with an executor [LUCENE-8727]

Open
#9,773 7 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

If IndexSearcher is configured with an executor, then the top docs for each slice are computed separately before being merged once the top docs for all slices are computed. With block-max WAND this is a bit of a waste of resources: it would be better if an increase of the min competitive score could help skip non-competitive hits on every slice and not just the current one.

---
Migrated from [LUCENE-8727](https://issues.apache.org/jira/browse/LUCENE-8727) by Adrien Grand (@jpountz), updated Oct 01 2019

Contributor guide

Open the contributing guide

Research direction

Start at IndexSearcher#search(Query,int) and trace how executor-backed slices compute and merge their top docs. Inspect the block-max WAND min-competitive-score flow across slices; done means a shared priority queue lets score increases from one slice help skip non-competitive hits in every slice.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.