apache / apache/lucene

Weird Performance regression due to: #13907

Open
#14,281 1 comment 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Elasticsearch's nightly benchmarks noticed a strange regression where the entire merge time for one of our vector benchmarks almost doubled in size. (Note the cumulative merge time graphed here: https://elasticsearch-benchmarks.elastic.co/#tracks/so_vector/nightly/default/90d)

After some digging and testing (manually running different lucene patches on the same hardware), I verified it was due to https://github.com/apache/lucene/pull/13907

I am fairly ignorant of the impact of this change, nor what the expected impact should be.

Maybe Elasticsearch is providing poor settings to Lucene and didn't handle the change correctly?

Or the change itself made a larger impact that originally designed?

### Version and environment details

Lucene 10.1

Intel IceLake CPU

8 vCPUs, 16GB RAM and 1x300GiB SSD disk

Contributor guide

Open the contributing guide

Research direction

Start with Lucene pull request #13907 and compare its behavior against the Elasticsearch nightly vector benchmark linked in the issue. Reproduce the merge-time regression on the stated Lucene 10.1, IceLake, 8-vCPU environment, then determine whether the result comes from settings or from the change itself. Done means the cause and expected performance impact are documented or corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.