apache / apache/lucene

Multi-Threading in IndexSearcher (after removal of MultiSearcher and ParallelMultiSearcher) [LUCENE-2840]

Open
#3,914 9 comments 0 reactions 0 assignees View on GitHub
legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-priority:Minor module:core/search type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Spin-off from parent issue:

> We should discuss about how many threads should be spawned\. If you have an index with many segments, even small ones, I think only the larger segments should be separate threads, all others should be handled sequentially\. So maybe add a maxThreads cound, then sort the IndexReaders by maxDoc and then only spawn maxThreads-1 threads for the bigger readers and then one additional thread for the rest?

---
Migrated from [LUCENE-2840](https://issues.apache.org/jira/browse/LUCENE-2840) by Uwe Schindler (@uschindler), updated May 09 2016
Parent: #3911

Contributor guide

Open the contributing guide

Research direction

Start with the parent issue #3911 and the migrated LUCENE-2840 discussion. Clarify the intended threading model for IndexSearcher, including maxThreads, segment ordering, and how smaller segments are handled; the issue is not ready to define a concrete implementation or completion test.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.