apache / apache/lucene

Can IndexSearcher enable search concurrency by default? [LUCENE-10487]

Open
#11,523 2 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

IndexSearcher has the ability to search an index using multiple threads, but this behavior is an opt-in. Could we enable it by default so that:
 - Users experience better latency out-of-the-box.
 - Lucene performs better in comparative benchmarks against other engines that leverage multiple threads by default.

Note that this isn't totally straightforward to implement, as we'd need to share the threadpool across all index searchers and think of when to shut down this threadpool?

---
Migrated from [LUCENE-10487](https://issues.apache.org/jira/browse/LUCENE-10487) by Adrien Grand (@jpountz)

Contributor guide

Open the contributing guide

Research direction

Start at the IndexSearcher entry point and trace how its existing opt-in multi-threaded search is configured. Investigate how a thread pool could be shared across index searchers and when it should shut down; done means concurrent search is enabled by default with a defined lifecycle and no regressions in search behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.