apache / apache/lucene

Reconsider boolean max clause exception [LUCENE-2018]

Open
#3,093 8 comments 0 reactions 0 assignees View on GitHub
legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-priority:Major module:core/search type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Now that we have smarter multi-term queries, I think its time to reconsider the boolean max clause setting. It made more sense before, because you could hit it more unaware when the multi-term queries got huge - now its more likely that if it happens its because a user built the boolean themselves. And no duh thousands more boolean clauses means slower perf and more resources needed. We don't throw an exception when you try to use a ton of resources in a thousand other ways.

The current setting also suffers from the static hell argument - especially when you consider something like Solr's multicore feature - you can have different settings for this in different cores, and the last one is going to win. Its ugly. Yes, that could be addressed better in Solr as well - but I still think it should be less ugly in Lucene as well.

I'd like to consider either doing away with it, or raising it by quite a bit at the least. Or an alternative better solution. Right now, it aint so great.

---
Migrated from [LUCENE-2018](https://issues.apache.org/jira/browse/LUCENE-2018) by Mark Miller (@markrmiller), updated May 09 2016

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points and presents several alternatives rather than a settled change. Start by locating the current boolean max-clause setting and its tests, then review the linked LUCENE-2018 discussion. Done requires an agreed behavior and corresponding implementation and regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.