Could bpVector reordering be used to make HNSW behave better in low memory?
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Since the vector reordering attempts to sort vectors on disk by their distances, it makes me wonder if it can improve HNSW's low memory latency (or even just latency as a whole).
Now that we have the "bulkScore" interface for scoring a set of candidates a time, we could:
- prefetch vectors
- then bulk score
the bpvector reordering thing makes this more interesting as prefetch could pull in vectors (by "accident") that will be scored soon (as vectors near eachother are close together on disk).
Maybe this is just a crazy idea.
Contributor guide
Research direction
Start by reading the existing bpVector reordering implementation and the HNSW bulkScore interface, then trace how candidate vectors are prefetched and scored. The issue names no files or tests, so first identify the relevant search and indexing entry points. Done would require a measured decision, supported by low-memory latency benchmarks, on whether reordering and prefetching improve HNSW behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100