apache / apache/lucene

Are we spending too much CPU in HNSW `ramBytesUsed`, and clearing bitsets?

Open
#14,763 6 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Peeking at the last nightly benchmark data point, I see the [top CPU hotspots during indexing](https://benchmarks.mikemccandless.com/2025.06.07.18.05.10.html#profiler_deterministic_indexing_4_cpu).

That 2nd one (13.12% in `ramBytesUsed`) is concerning ... I know we need to account properly for RAM so IW can flush when RAM exceeds its allowance ... but maybe we can optimize how we do that for HNSW?

8.63% spent clearing bitsets for HNSW searching is also scary -- that likely impacts search performance too (since building an HNSW graph is done by doing a search for each inserted vector)?

Also what exactly is `reduceLanesTemplate`? I find this name very non-intuitive :) Is it essentially a cast (like `long` -> `int`) for a vector?

```
Profiler for cpu:
WARNING: Using incubator modules: jdk.incubator.vector
PROFILE SUMMARY from 4433882 events (total: 4M)
tests.profile.mode=cpu
tests.profile.count=50
tests.profile.stacksize=4
tests.profile.linenumbers=false
PERCENT CPU SAMPLES STACK
23.57% 1M jdk.incubator.vector.FloatVector#reduceLanesTemplate() [Inlined code]
at jdk.incubator.vector.Float256Vector#reduceLanes() [Inlined code]
at org.apache.lucene.internal.vectorization.PanamaVectorUtilSupport#dotProductBody() [JIT compiled code]
at org.apache.lucene.internal.vectorization.PanamaVectorUtilSupport#dotProduct() [Inlined code]
13.12% 581719 org.apache.lucene.util.hnsw.NeighborArray#ramBytesUsed() [Inlined code]
at org.apache.lucene.util.hnsw.OnHeapHnswGraph#updateGraphRamBytesUsed() [JIT compiled code]
at org.apache.lucene.util.hnsw.OnHeapHnswGraph#addNode() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNodeInternal() [JIT compiled code]
9.51% 421729 org.apache.lucene.index.FloatVectorValues$1#vectorValue() [Inlined code]
at org.apache.lucene.codecs.hnsw.DefaultFlatVectorScorer$FloatScoringSupplier$1#score() [Inlined code]
at org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNodeInternal() [JIT compiled code]
8.63% 382600 java.util.Arrays#fill() [Inlined code]
at org.apache.lucene.util.FixedBitSet#clear() [Inlined code]
at org.apache.lucene.util.hnsw.HnswGraphSearcher#prepareScratchState() [Inlined code]
at org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel() [JIT compiled code]
4.55% 201882 org.apache.lucene.util.FixedBitSet#getAndSet() [Inlined code]
at org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNodeInternal() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNode() [Inlined code]
3.12% 138341 org.apache.lucene.util.RamUsageEstimator#sizeOf() [Inlined code]
at org.apache.lucene.internal.hppc.MaxSizedIntArrayList#ramBytesUsed() [Inlined code]
at org.apache.lucene.util.hnsw.NeighborArray#ramBytesUsed() [Inlined code]
at org.apache.lucene.util.hnsw.OnHeapHnswGraph#updateGraphRamBytesUsed() [JIT compiled code]
2.60% 115254 org.apache.lucene.util.hnsw.HnswConcurrentMergeBuilder$MergeSearcher#graphSeek() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNodeInternal() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNode() [Inlined code]
2.42% 107338 org.apache.lucene.internal.hppc.MaxSizedIntArrayList#ramBytesUsed() [Inlined code]
at org.apache.lucene.util.hnsw.NeighborArray#ramBytesUsed() [Inlined code]
at org.apache.lucene.util.hnsw.OnHeapHnswGraph#updateGraphRamBytesUsed() [JIT compiled code]
at org.apache.lucene.util.hnsw.OnHeapHnswGraph#addNode() [JIT compiled code]
2.36% 104842 org.apache.lucene.internal.vectorization.PanamaVectorUtilSupport#dotProductBody() [JIT compiled code]
at org.apache.lucene.internal.vectorization.PanamaVectorUtilSupport#dotProduct() [Inlined code]
at org.apache.lucene.util.VectorUtil#dotProduct() [Inlined code]
at org.apache.lucene.index.VectorSimilarityFunction$2#compare() [Inlined code]
2.36% 104836 org.apache.lucene.util.hnsw.OnHeapHnswGraph#updateGraphRamBytesUsed() [JIT compiled code]
at org.apache.lucene.util.hnsw.OnHeapHnswGraph#addNode() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNodeInternal() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNode() [Inlined code]
2.21% 97814 org.apache.lucene.util.hnsw.OnHeapHnswGraph#getNeighbors() [Inlined code]
at org.apache.lucene.util.hnsw.HnswConcurrentMergeBuilder$MergeSearcher#graphSeek() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel() [JIT compiled code]
at org.apache.lucene.util.hnsw.HnswGraphBuilder#addGraphNodeInternal() [JIT compiled code]
1.55% 68929 java.util.concurrent.locks.AbstractQueuedSynchronizer#apparentlyFirstQueuedIsExclusive() [Inlined code]
at java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync#readerShouldBlock() [Inlined code]
at java.util.concurrent.locks.ReentrantReadWriteLock$Sync#tryAcquireShared() [Inlined code]
at java.util.concurrent.locks.AbstractQueuedSynchronizer#acquireShared() [Inlined code]

```

Contributor guide

Open the contributing guide

Research direction

Start with the linked nightly CPU profile, then inspect NeighborArray#ramBytesUsed, OnHeapHnswGraph#updateGraphRamBytesUsed, HnswGraphSearcher#prepareScratchState, and PanamaVectorUtilSupport#reduceLanesTemplate callers. The issue does not define a specific change or acceptance test; completion would require an agreed optimization target backed by benchmark evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.