apache / apache/lucene

Performance regression starting in Lucene 10.1.0 (dis_max + constant_score)

Open
#15,658 1 comment 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

I observed a performance regression after upgrading Elasticsearch 8.19.8 (Lucene 9.12.3) to Elasticsearch 9.2.4 (Lucene 10.3.2). The slowdown appears for a query shape that uses dis_max + constant_score. I didn’t find any ES‑level changes that explain the regression, so I suspect Lucene.

After benchmarking performance of a query on multiple Lucene version, I confirmed a regression starting in Lucene 10.1.0.

__Query shape__

- dis_max (tie=0)
- boost 25 * constant_score( OR of term queries on field1 + field2, minShouldMatch=1 )
- boost 10 * constant_score( OR of term queries on field3, minShouldMatch=1 )

__Reproducer__

I created small standalone Lucene benchmark that reproduce performance regression: https://github.com/shimpeko/lucene-dismax-constantscore-benchmark

__Benchmark result (Java 25.0.2, 5k docs, warmup 200, iters 1000)__

- 9.12.3 -> 0.424 ms/iter
- 10.0.0 -> 0.439 ms/iter
- 10.1.0 -> 1.167 ms/iter
- 10.2.0 -> 1.312 ms/iter
- 10.2.1 -> 1.335 ms/iter
- 10.2.2 -> 1.345 ms/iter
- 10.3.0 -> 1.377 ms/iter
- 10.3.1 -> 1.335 ms/iter
- 10.3.2 -> 1.302 ms/iter

The regression starts at 10.1.0 and persists in later 10.x. There is also miner regression on 10.2.0

### Version and environment details

__Environment__

- OS: macOS 26.2 (Build 25C56), Darwin 25.2.0
- CPU: Apple M1 Max (10 cores: 8 performance + 2 efficiency)
- RAM: 64 GB
- Java: 25.0.2 (OpenJDK)
- Lucene directory: ByteBuffersDirectory
- Analyzer: WhitespaceAnalyzer
- Index: 5,000 docs, single segment (forceMerge(1))
- Searcher: single‑threaded (no executor)

__Notes__
- In ES tests, similar query without constant_score or without dis_max did not show notable regress.

Let me know if you’d like additional data or a different reproduction setup.

Contributor guide

Open the contributing guide

Research direction

Start by running the standalone benchmark linked in the issue and compare Lucene 10.0.0 with 10.1.0 using the reported dis_max and constant_score query shape. Inspect the Lucene changes between those versions, then validate the suspected cause against the later 10.x timings. Done means the regression source is identified and a fix or confirmed explanation is demonstrated with the reproducer.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.