NVIDIA / NVIDIA/cuvs

[BUG] t thread blocks inside cuvs::neighbors::dynamic_batching::search when conservative_dispatch = true

Open
#2,081 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

Describe the bug
A clear and concise description of what the bug is.

Every client thread blocks inside cuvs::neighbors::dynamic_batching::search
and never returns. The dispatch_timeout_ms codepath does not fire, so even
when the batch can never fill (concurrent_clients < max_batch_size), the
dispatcher never falls back to launching the upstream search at the partial
size. The program hangs indefinitely.

Configuration that reproduces it on our host (one RTX 5070 Laptop GPU,
CUDA 13.0, libcuvs 26.02.000 / libraft 26.02.000):

 max_batch_size       = 4
 n_queues             = 3
 conservative_dispatch= true
 dispatch_timeout_ms  = 100   (fast for the test; doesn't fire either way)
 concurrent clients   = 16    (>> max_batch_size, but each client opens
                               its own raft::resources / stream, so the
                               wrapper splits them across queues and no
                               single queue ever sees max_batch_size in
                               flight at once)

The same program with conservative_dispatch = false (the cuVS default) runs
to completion. That difference is the bug.

Steps/Code to reproduce bug
Follow this guide http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports to craft a minimal bug report. This helps us reproduce the issue you're having and resolve the issue more quickly.

Yes, program attached.

Expected behavior
A clear and concise description of what you expected to happen.

conservative_dispatch=true should not hang or deadlock.

Environment details (please complete the following information):

  • Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
  • Method of RAFT install: [conda, Docker, or from source]
    • If method of install is [Docker], provide docker pull & docker run commands used

Configuration that reproduces it on our host (one RTX 5070 Laptop GPU,
CUDA 13.0, libcuvs 26.02.000 / libraft 26.02.000):
OS: Windows WSL2 (Only reproducible in WSL2)

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at cuvs::neighbors::dynamic_batching::search and trace the conservative_dispatch=true path, especially dispatch_timeout_ms handling across raft::resources, streams, and queues. Reproduce the WSL2 configuration with max_batch_size=4 and n_queues=3; done means partial batches eventually launch or return without hanging, while conservative_dispatch=false behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.