NVIDIA / NVIDIA/cuvs

[BUG] NNTest: GTEST_SKIP() in compute_1nn() doesn't skip the test, so cuTile-backend cases fail when cuTile is unavailable

Open Beginner friendly
#2,660 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

Related: #2552 (introduced the cuTile top-1 NN backend and these tests)

Summary

In cpp/tests/neighbors/distance_nn.cu, the cuTile availability check calls
GTEST_SKIP() from inside the helper compute_1nn():

void compute_1nn()                                    // line 120
{
  ...
  if (backend == cuvs::distance::detail::Top1nnBackend::Cutile &&
      !cuvs::distance::detail::is_top_1_nn_backend_available(
        backend, x.data_handle(), y.data_handle(), m, n, k, metric)) {
    GTEST_SKIP() << "cuTile is not available for this device/input";   // line 132
  }

GTEST_SKIP() only returns from the function it is written in. Every TEST_P
body is this->compute_1nn(); this->compare(); (lines 294-395), so after the
"skip" the test carries on into compare(), which then checks output buffers
that were never written, and fails. The result is a [ SKIPPED ]-style message
immediately followed by a spurious [ FAILED ] for the same test.

This only shows up where the cuTile backend is compiled in but reports
unavailable at runtime, so it is likely invisible on devices where cuTile works.

Environment
  • NVIDIA GB10 (DGX Spark, aarch64), compute capability 12.1, driver 580.173.02
  • CUDA 13.4, CUVS_CUTILE_ENABLED
  • cuvs e0f8a4eb; upstream main 1943ca8d has identical code at the lines above
  • On this machine is_top_1_nn_backend_available(Cutile, ...) is false for every
    cuTile-backend case (why it is unavailable here is not relevant to the bug)
Reproduce
./NEIGHBORS_TEST --gtest_filter='NNTest/NNTest_fp32_fused.test/16'
[ RUN      ] NNTest/NNTest_fp32_fused.test/16
.../distance_nn.cu:132: Skipped
cuTile is not available for this device/input

.../distance_nn.cu:205: Failure
Value of: cuvs::devArrMatch(ref_dist.data_handle(), selected_dist.data_handle(), ...)
  Actual: false (actual=55.788799285888672 != expected=22.509374618530273 @0)
[  FAILED  ] NNTest/NNTest_fp32_fused.test/16

12 cases fail this way: NNTest_fp32_fused.test/16..24 (9), NNTest_fp16_fused.test/0..1
(2) and NNTest_fp32_fused_i64.test/0 (1). That is exactly the number of "cuTile is
not available" messages in the run (12). The mismatching values look like
"wrong results" but are just unwritten outputs: different shapes report the same
value, and no cuTile kernel ran.

Suggested fix

Return early from compare() when the test was skipped:

void compare()
{
  // GTEST_SKIP() in compute_1nn() only returns from that helper, not the test body.
  if (::testing::Test::IsSkipped()) { return; }
  ...

With this change on the same machine, NNTest/* gives 42 passed, 12 skipped,
0 failed (previously 42 passed, 12 failed). We have not tested alternatives such
as returning early in each TEST_P body or moving the availability check into
SetUp() (where GTEST_SKIP() does skip the whole test).


Assisted-by: Claude

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open cpp/tests/neighbors/distance_nn.cu and inspect compute_1nn(), compare(), and the NNTest parameterized bodies around the cited lines. Reproduce with ./NEIGHBORS_TEST --gtest_filter='NNTest/NNTest_fp32_fused.test/16', then ensure unavailable cuTile cases do not continue into output comparison. Run the NNTest suite and confirm 42 passed, 12 skipped, and 0 failed.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing-qa
Issue type
Bug
Difficulty
1/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.