apache / apache/lucene

Understand 2025/08/06 nightly benchy regression in KNN indexing

Open
#15,079 50 comments 0 reactions 1 assignee Claimed by @uschindler View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

[Spinoff from [this comment](https://github.com/apache/lucene/pull/14932#issuecomment-3167938331)]

We recently merged a [nice opto to use GroupVarInt instead of VarInt for encoding each node's neighbors list in the HNSW (KNN) graph](https://github.com/apache/lucene/pull/14932). Benchmarks showed improvements to search latency, which we expect since GroupVarInt decodes four ints at once and puts less branch prediction stress on the CPU.

But oddly nightly vector indexing benchmarks seemed to slow down, on 2025/08/06, and it's not yet obvious why -- it isn't one of the random "glitches" that that recover after a day or two (those are separately annoying! [`beast3` benchy box is old now](https://blog.mikemccandless.com/2021/01/apache-lucene-performance-on-128-core.html), ~4.5 years, which equates to 135 years in humans years! (1 computer year = 30 human years) Maybe it is getting a bit flakey in its later years...).

I'll try to dig some more ... but if I get nowhere (likely!), I think we should temporarily revert GroupVarInt and see if benchy recovers? Especially because this might become a blocker for 10.3 release...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.