apache / apache/lucene

Improve exception handling for concurrent slice search

Open
#12,278 0 comments 1 reaction 0 assignees View on GitHub
discussion
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

When searching concurrently across slices, an exception may be thrown by one of the collectors. In that case, an `ExecutionException` will be thrown when `Future#get` is called. The `search` method will wrap and re-throw the exception once the first future in the list throws one. The collectors for the other slices will keep on running until completion, but their results will be entirely ignored.

Is it a reasonable expectation that once `IndexSearcher#search(Query, CollectorManager)` returns, that specific search has completed and has left no threads behind still working on it? I would expect this to be true, but that is not the case today in case an exception is thrown. It may just mean waiting for all slices to be completed before re-throwing the exception, although that means possibly waiting for results that are completely ignored.

Instead of completing ongoing work for nothing, when there's an exception it sounds like it would be a good idea to stop ongoing work on the remaining collectors? Could we perhaps try and reuse/adapt the timeout mechanism that `IndexSearcher` supports so that we can at least notify the `BulkScorer` that it should stop its work? This would require wrapping the bulk scorer at all times when searching across multiple slices in parallel, regardless of whether a timeout has been set or not, which sounds reasonable to me.

Contributor guide

Open the contributing guide

Research direction

Start with IndexSearcher#search(Query, CollectorManager) and trace how Future#get handles an ExecutionException while other slice collectors continue running. Compare the existing timeout mechanism and BulkScorer behavior, then define and test completion semantics so returning from the search does not leave work from that search running.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.