NVIDIA / NVIDIA/cuvs

[FEA] Support Mutual Reachability distance for ANN index types

Open
#304 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

Mutual reachability distance is an important post-processing data mining computation that unfortunately needs to be computed during the distance computation. Because RAFT is header only, we exposed this capability as a distance epilogue function that could be applied in place just after the actual distance is computed for brute force (and more recently for nn-descent).

CuVS is not header only, and so this poses additional challenges in specifying mutual reachability using a generalized epilogue.

Our solution is to have cuVS support the mutual reachability distance directly, since it's pretty general. The difference between mutual reachability and other more common distance types is that it requires having an array of "the distance to the kth neighbor" available (referred to formally as core distances).

We should be able to support this distance type without too much trouble, but will probably need to figure out what the API should look like. I'm marking this as a placeholder feature, since @benfred plans to take a look at migrating this to cuVS so we can formally move cuML over to using cuVs Instead of RAFT for nearest neighbors things.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the existing distance-epilogue support for brute force and nn-descent, then examine the cuVS nearest-neighbor APIs involved in migrating this capability from RAFT. Define an API for supplying core distances and supporting mutual reachability across ANN index types; the issue names no files or tests, so completion criteria still need to be established.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.