[CI] SCANN Test flaky
Open
Nobody has claimed this yet.
bug
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Noticed that NEIGHBORS_ANN_SCANN_TEST failed:
Full logs here: https://github.com/NVIDIA/cuvs/actions/runs/33689912368/job/100455474437?pr=2264
Env:
- RTX Pro 6000
- CUDA 13.3.0,
- python 3.13,
- amd64
- ubuntu26.04
- latest-driver, latest-deps
[ RUN ] ScaNN/f32_i64.build_host_input/13
/tmp/conda-bld-output/bld/rattler-build_libcuvs-headers/work/cpp/tests/neighbors/ann_scann/../ann_scann.cuh:267: Failure
Expected: (max_error) < (max_allowed_error * 1.5f), actual: 1.43570065 vs 1.425
Max reconstruction error too large: 1.4357006549835205
[ FAILED ] ScaNN/f32_i64.build_host_input/13, where GetParam() = 80-byte object <00-10 00-00 08-00 00-00 00-00 00-00 00-00 00-40 20-00 00-00 00-00 00-00 00-10 00-00 00-00 00-00 18-00 00-00 00-00 80-3F 00-00 80-3F 02-00 00-00 08-00 00-00 00-00 00-00 00-10 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 C0-7F 00-00 00-00> (61 ms)
September 9th:
https://github.com/NVIDIA/cuvs/pull/2565#issuecomment-5612135083
[ RUN ] ScaNN/f32_i64.build_host_input/12
/tmp/conda-bld-output/bld/rattler-build_libcuvs-headers/work/cpp/tests/neighbors/ann_scann/../ann_scann.cuh:267: Failure
Expected: (max_error) < (max_allowed_error * 1.5f), actual: 1.43224525 vs 1.425
Max reconstruction error too large: 1.4322452545166016
[ FAILED ] ScaNN/f32_i64.build_host_input/12, where GetParam() = 80-byte object <00-10 00-00 06-00 00-00 00-00 00-00 00-00 00-40 20-00 00-00 00-00 00-00 00-10 00-00 00-00 00-00 18-00 00-00 00-00 80-3F 00-00 80-3F 02-00 00-00 08-00 00-00 00-00 00-00 00-10 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 C0-7F 00-00 00-00> (56 ms)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with cpp/tests/neighbors/ann_scann/ann_scann.cuh around line 267 and run the NEIGHBORS_ANN_SCANN_TEST using the reported environment. Compare the failing build_host_input cases and the logged max_error against max_allowed_error. Done means the test no longer flakes or the failure has a demonstrated, stable explanation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100