apache / apache/lucene

ParallelReader should support getSequentialSubReaders if possible [LUCENE-2766]

Open
#3,840 9 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major module:core/index type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Applications that need to use ParallelReader can't currently use per-segment optimizations because getSequentialSubReaders returns null.

Considering the strict requirements on input indexes that ParallelReader already enforces it's usually the case that the additional indexes are built with the knowledge of the primary index, in order to keep the docId-s synchronized. If that's the case then it's conceivable that these indexes could be created with the same number of segments, which in turn would mean that their docId-s are synchronized on a per-segment level. ParallelReader should detect such cases, and in getSequentialSubReader it should return an array of ParallelReader-s created from corresponding segments of input indexes.

---
Migrated from [LUCENE-2766](https://issues.apache.org/jira/browse/LUCENE-2766) by Andrzej Bialecki (@sigram), updated Jan 04 2011
Attachments: [LUCENE-2766.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2766/LUCENE-2766.patch)

Contributor guide

Open the contributing guide

Research direction

Start by reading ParallelReader and its getSequentialSubReaders method, then inspect the attached LUCENE-2766.patch for the historical implementation direction. Check how corresponding input indexes expose segments and verify the behavior when segment counts and docIds are synchronized; done means eligible cases return corresponding ParallelReader sub-readers while unsupported cases remain safe.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.