NVIDIA / NVIDIA/cuvs

[Tracker] Improving cuvs-lucene release robustness

Open
#2,512 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

Java compatibility and support

  • [26.12][^1] #2517
  • [26.12] Keep JDK 21 compile-time support and add a CI job that builds/runs on JDK 21
  • [26.12] Clearly document the JDK support policy (compile vs. run requirements)

Lucene support

  • [26.12] #2410 and expand with:
    1. Restructure cuvs-lucene code to support different Lucene API for each supported version
    2. restructure cuvs-lucene build to publish one JAR per Lucene minor (cuvs-lucene-10.N-<ver>.jar)
    3. Compile against lowest patch version, test against highest patch version per minor
  • [26.12] Define a policy for adding new Lucene minors and dropping trailing ones

libcuvs ↔ cuvs-java ↔ cuvs-lucene compatibility

  • #1682 (Document libcuvs ↔ cuvs-java ↔ cuvs-lucene compatibility rules)
  • #1020
  • Add a CI matrix testing supported libcuvs × cuvs-java × cuvs-lucene combinations

API compatibility

  • Publish an API compatibility policy for cuvs-java/cuvs-lucene (no guarantee across minors, Lucene-style)

Index file format compatibility

  • Publish an index format backward-compatibility guarantee (readable through the next major of lucene)
  • Start working on a corpus of indices from older releases and test reading them or add a harness to be able to generate old indices on demand.
  • Add backward-compatibility test matrix across Lucene versions × cuvs releases

Cross-platform compatibility

  • Add explicit runtime checks + error messages for missing CUDA/insufficient VRAM when opening GPU indices
  • Document platform/portability limits of GPU index files (snapshot/restore across clusters)

Performance guarantees

  • #2194
  • #1856
  • #1623
  • #2195
  • #2428
  • #2014
  • #2180 Ideally also stand up a benchmark-results dashboard
  • Expand benchmarks to run across different GPUs (RTX Pro 6000, H100, A10, L40/L40S) and multiple datasets

Testing

  • #2407 Expand with updates and deletes for some records
  • Expand reproducible randomized testing with seeds in cuvs-lucene, surface test seeds in CI logs and treat seed-specific failures as real bugs, not flaky
  • #2105

[^1]: Items marked with [26.12] are targeted for the [26.12] release.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by reviewing the linked issues (#2517, #2410, #1682, #1020, and the performance/testing issues), then group the checklist by compatibility, release, performance, and testing work. This tracker is done when its listed cuvs-lucene robustness requirements are completed and each checklist item is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, ci-cd, documentation, performance, search, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.