[BUG] [Java] `CagraRandomizedIT.testResultsTopKWithRandomValues` fails randomly
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
CagraRandomizedIT.testResultsTopKWithRandomValues fails occasionally, causing CI pipelines to fail randomly.
This was seen on the second-last CI run on #1366. I was able to repro this locally:
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.481 s <<< FAILURE! -- in com.nvidia.cuvs.CagraRandomizedIT
[ERROR] com.nvidia.cuvs.CagraRandomizedIT.testResultsTopKWithRandomValues -- Time elapsed: 3.479 s <<< FAILURE!
java.lang.AssertionError: Not found in expected list: 448
at __randomizedtesting.SeedInfo.seed([DD841DFC69E8C00:24F49D4798CD7240]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at com.nvidia.cuvs@25.12.0/com.nvidia.cuvs.CuVSTestCase.compareResults(CuVSTestCase.java:104)
at com.nvidia.cuvs@25.12.0/com.nvidia.cuvs.CagraRandomizedIT.tmpResultsTopKWithRandomValues(CagraRandomizedIT.java:170)
at com.nvidia.cuvs@25.12.0/com.nvidia.cuvs.CagraRandomizedIT.testResultsTopKWithRandomValues(CagraRandomizedIT.java:44)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1763)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:426)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:716)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:637)
It would be good to get to the bottom of this. To avoid further churn in CI, it might be good to temporarily back this test out.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with CagraRandomizedIT.testResultsTopKWithRandomValues and its helper at CagraRandomizedIT.java:170, then inspect CuVSTestCase.compareResults at line 104 using the reported randomized seed. Determine why expected results intermittently omit a returned value, then stabilize the test or temporarily back it out so CI no longer fails randomly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100