apache / apache/lucene

Should ExitableTermsEnum wrap postings and impacts? [LUCENE-10544]

Open
#11,580 14 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major module:core/index type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

While looking into options for #11188, I noticed that `ExitableDirectoryReader` doesn't actually do any timeout checking once you start iterating postings/impacts. It **does** create a `ExitableTermsEnum` wrapper when loading a `TermsEnum`}, but that wrapper doesn't do anything to wrap postings or impacts. So timeouts will be enforced when moving to the "next" term, but not when iterating the postings/impacts associated with a term.

I think we ought to wrap the postings/impacts as well with some form of timeout checking so timeouts can be enforced on long-running queries. I'm not sure why this wasn't done originally (back in 2014), but it was questioned back in 2020 on the original Jira [SOLR-5986](https://issues.apache.org/jira/browse/SOLR-5986). Does anyone know of a good reason why we shouldn't enforce timeouts in this way?

Related, we may also want to wrap things like `seekExact` and `seekCeil` given that only `next` is being wrapped currently.

---
Migrated from [LUCENE-10544](https://issues.apache.org/jira/browse/LUCENE-10544) by Greg Miller (@gsmiller), updated Jul 29 2022

Contributor guide

Open the contributing guide

Research direction

Start by reading ExitableDirectoryReader and ExitableTermsEnum, focusing on the TermsEnum.next behavior and the postings, impacts, seekExact, and seekCeil paths named in the issue. Done means the timeout behavior for these iteration and seek operations is resolved and verified, with the appropriate tests identified in the surrounding Lucene code.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.