NVIDIA / NVIDIA/cuvs

[BUG] cuvs-java com.nvidia.cuvs.CagraRandomizedTest.testResultsTopKWithRandomValues test fails intermittently

Open
#631 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

Describe the bug
cuvs-java test fails intermittently, e.g.

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.031 s <<< FAILURE! -- in com.nvidia.cuvs.CagraRandomizedTest
[ERROR] com.nvidia.cuvs.CagraRandomizedTest.testResultsTopKWithRandomValues -- Time elapsed: 2.031 s <<< FAILURE!
java.lang.AssertionError: Not found in expected list: 1583
  at __randomizedtesting.SeedInfo.seed([93DF604C314D221A:BAF3BCD46F1EDC5A]:0)
  at org.junit.Assert.fail(Assert.java:89)
  at org.junit.Assert.assertTrue(Assert.java:42)
  at com.nvidia.cuvs/com.nvidia.cuvs.CuVSTestCase.compareResults(CuVSTestCase.java:83)
  at com.nvidia.cuvs/com.nvidia.cuvs.CagraRandomizedTest.tmpResultsTopKWithRandomValues(CagraRandomizedTest.java:98)
  at com.nvidia.cuvs/com.nvidia.cuvs.CagraRandomizedTest.testResultsTopKWithRandomValues(CagraRandomizedTest.java:29)
  at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
  at java.base/java.lang.reflect.Method.invoke(Method.java:580)
...

Steps/Code to reproduce bug
It's not clear if setting the random seed (-Dtests.seed) actually improves reproducibility.

./build java
cd java/cuvs-java
mvn -Dtests.seed=93DF604C314D221A test

Expected behavior
The test passes successfully every time.

Environment details (please complete the following information):

  • Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
    • AWS g4dn.2xlarge, Ubuntu 22.04 + cuda 12.6
  • Method of RAFT install: [conda, Docker, or from source]
    • If method of install is [Docker], provide docker pull & docker run commands used

set the JAVA_HOME env variable to JDK 22

JAVA_HOME=/home/ubuntu/binaries/jdk-22.0.2/

Additional context
This fails in branch-25.02, with the commit that added the java-api:

commit 4ca47c92b03856e7d4ae328aabf6ee3e853a137e (...

Initial cut for a cuVS Java API (#450)
    
A Java API for cuVS for easy integration into Apache Lucene or other Java based projects.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in java/cuvs-java with com.nvidia.cuvs.CagraRandomizedTest.testResultsTopKWithRandomValues and the failure path in CuVSTestCase.compareResults. Build with ./build java, then run mvn -Dtests.seed=93DF604C314D221A test from java/cuvs-java on the documented CUDA environment. Done means the randomized test passes reliably, including with the reported seed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.