[BUG] NNTest: GTEST_SKIP() in compute_1nn() doesn't skip the test, so cuTile-backend cases fail when cuTile is unavailable
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Related: #2552 (introduced the cuTile top-1 NN backend and these tests)
Summary
In cpp/tests/neighbors/distance_nn.cu, the cuTile availability check calls
GTEST_SKIP() from inside the helper compute_1nn():
void compute_1nn() // line 120
{
...
if (backend == cuvs::distance::detail::Top1nnBackend::Cutile &&
!cuvs::distance::detail::is_top_1_nn_backend_available(
backend, x.data_handle(), y.data_handle(), m, n, k, metric)) {
GTEST_SKIP() << "cuTile is not available for this device/input"; // line 132
}
GTEST_SKIP() only returns from the function it is written in. Every TEST_P
body is this->compute_1nn(); this->compare(); (lines 294-395), so after the
"skip" the test carries on into compare(), which then checks output buffers
that were never written, and fails. The result is a [ SKIPPED ]-style message
immediately followed by a spurious [ FAILED ] for the same test.
This only shows up where the cuTile backend is compiled in but reports
unavailable at runtime, so it is likely invisible on devices where cuTile works.
Environment
- NVIDIA GB10 (DGX Spark, aarch64), compute capability 12.1, driver 580.173.02
- CUDA 13.4,
CUVS_CUTILE_ENABLED - cuvs
e0f8a4eb; upstream main1943ca8dhas identical code at the lines above - On this machine
is_top_1_nn_backend_available(Cutile, ...)is false for every
cuTile-backend case (why it is unavailable here is not relevant to the bug)
Reproduce
./NEIGHBORS_TEST --gtest_filter='NNTest/NNTest_fp32_fused.test/16'
[ RUN ] NNTest/NNTest_fp32_fused.test/16
.../distance_nn.cu:132: Skipped
cuTile is not available for this device/input
.../distance_nn.cu:205: Failure
Value of: cuvs::devArrMatch(ref_dist.data_handle(), selected_dist.data_handle(), ...)
Actual: false (actual=55.788799285888672 != expected=22.509374618530273 @0)
[ FAILED ] NNTest/NNTest_fp32_fused.test/16
12 cases fail this way: NNTest_fp32_fused.test/16..24 (9), NNTest_fp16_fused.test/0..1
(2) and NNTest_fp32_fused_i64.test/0 (1). That is exactly the number of "cuTile is
not available" messages in the run (12). The mismatching values look like
"wrong results" but are just unwritten outputs: different shapes report the same
value, and no cuTile kernel ran.
Suggested fix
Return early from compare() when the test was skipped:
void compare()
{
// GTEST_SKIP() in compute_1nn() only returns from that helper, not the test body.
if (::testing::Test::IsSkipped()) { return; }
...
With this change on the same machine, NNTest/* gives 42 passed, 12 skipped,
0 failed (previously 42 passed, 12 failed). We have not tested alternatives such
as returning early in each TEST_P body or moving the availability check into
SetUp() (where GTEST_SKIP() does skip the whole test).
Assisted-by: Claude
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open cpp/tests/neighbors/distance_nn.cu and inspect compute_1nn(), compare(), and the NNTest parameterized bodies around the cited lines. Reproduce with ./NEIGHBORS_TEST --gtest_filter='NNTest/NNTest_fp32_fused.test/16', then ensure unavailable cuTile cases do not continue into output comparison. Run the NNTest suite and confirm 42 passed, 12 skipped, and 0 failed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100