Maybe Improve join block Vector search performance by block scoring child vectors
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Besides the overall performance improvements that could be done for HNSW & block join queries, I think there are ways for us to improve the vector story as a whole.
A major one is scoring ALL child docs for a nearest parent doc at a time. When we score candidates, we score the candidates together that are children of the same parent, and ALSO score ALL children for that common parent.
This would be complicated, a POC would likely be required to prove if its useful, but it would allow us to:
- Bulk score all matching children in a block (super fast, locality on disk, etc.)
- Bulk collect the children all within a parent, keeping the translation times simple
- It MAY increase the scoring count (e.g. now for a single node in the graph, we might score 10s or 100s of vectors :/), so maybe its something that only occurs once we get further into the graph...
Its logical that children are all near each other in the graph.
This will be a pretty large digression in the API design. The KnnCollector would need to:
- Provide the scoring logic (but not the score methodology)
- Keep track of "visited" nodes
This would also give some neat augmentations, like the ability to return the average score, min score, max score for a parent and more than the single nearest vector (e.g. could return the top 5 or whatever).
Contributor guide
Research direction
Start by examining the HNSW and block join query paths together with the KnnCollector API. Build a proof of concept to evaluate scoring all matching child vectors for a parent, including bulk collection and visited-node tracking. Done means measured evidence that the approach improves vector search performance without unacceptable scoring overhead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100