Can IndexSearcher enable search concurrency by default? [LUCENE-10487]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
IndexSearcher has the ability to search an index using multiple threads, but this behavior is an opt-in. Could we enable it by default so that:
- Users experience better latency out-of-the-box.
- Lucene performs better in comparative benchmarks against other engines that leverage multiple threads by default.
Note that this isn't totally straightforward to implement, as we'd need to share the threadpool across all index searchers and think of when to shut down this threadpool?
---
Migrated from [LUCENE-10487](https://issues.apache.org/jira/browse/LUCENE-10487) by Adrien Grand (@jpountz)
Contributor guide
Research direction
Start at the IndexSearcher entry point and trace how its existing opt-in multi-threaded search is configured. Investigate how a thread pool could be shared across index searchers and when it should shut down; done means concurrent search is enabled by default with a defined lifecycle and no regressions in search behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100