Lucene: investigate and address heuristics perf wrt specific dataset
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Reproducer:
- Check out latest main branches of cuvs and cuvs-lucene
- upload dataset alpha-123 (please ping me for access -- note requires NVIDIA access -- note alpha-123 is not the real name of dataset)
- run with automatic heuristics
This should reproduce the green line with poor search latency performance.
My hypothesis: I suspect the intrinsic dimension of the manifold within 1536d dataset isn't large enough to produce a preponderance of hub-vectors to account for the fact that we are running the benchmark on a baselayer-only graph.
What is apparent from extra tests: when I bypass the automatic heuristics with my own custom parameters, I get even worse search latency performance than the performance achieved by the automatic heuristics when I set cagraHnswLayers=1, but then the search-latency performance gradually increases until we basically match the expected search-latency at cagraHnswLayers=6.
Question: why can't we just increase the number of cagraHnswLayers while using the automatic heuristics?
Answer: we hit the "not enough KNN neighbors" RAFT errors when we try to increase the number of cagraHnswLayers above 1 (note: I typically see that behavior when nProbes is too low or nLists is too high)
Note: FAISS implements both of their CAGRA to HNSW conversions differently from cuvs-lucene, so this might only be an issue with cuvs-lucene?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by checking out the latest main branches of cuvs and cuvs-lucene and reproducing the benchmark with the alpha-123 dataset; access requires NVIDIA approval. Compare automatic heuristics with different cagraHnswLayers, nProbes, and nLists values, noting the RAFT “not enough KNN neighbors” errors. Done means identifying why the automatic configuration cannot use higher layers and documenting a validated fix or follow-up.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning, performance, search
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100