Should we use a FixedBitSet to track visited nodes in HnswGraph.search? [LUCENE-9847]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Let's compare the cost/benefit tradeoff of using full BitSet in HnswGraph.search. This idea was motivated by the large number of heap allocations due to SparseFixedBitSet used in graph search that we see in profiler output. We could manage a pool of full BitSets in order to minimize allocations there, and probably gain some performance with a modest expenditure of heap.
---
Migrated from [LUCENE-9847](https://issues.apache.org/jira/browse/LUCENE-9847) by Michael Sokolov (@msokolov)
Contributor guide
Research direction
Start by inspecting HnswGraph.search and the existing SparseFixedBitSet usage described in the issue. Compare allocation and search-performance tradeoffs for a pooled full BitSet, with the work complete when the measurement-based recommendation and implementation scope are established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance, search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100