[Tracker] Improving cuvs-lucene release robustness
Open
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Java compatibility and support
- [26.12][^1] #2517
- [26.12] Keep JDK 21 compile-time support and add a CI job that builds/runs on JDK 21
- [26.12] Clearly document the JDK support policy (compile vs. run requirements)
Lucene support
- [26.12] #2410 and expand with:
- Restructure cuvs-lucene code to support different Lucene API for each supported version
- restructure cuvs-lucene build to publish one JAR per Lucene minor (
cuvs-lucene-10.N-<ver>.jar) - Compile against lowest patch version, test against highest patch version per minor
- [26.12] Define a policy for adding new Lucene minors and dropping trailing ones
libcuvs ↔ cuvs-java ↔ cuvs-lucene compatibility
- #1682 (Document libcuvs ↔ cuvs-java ↔ cuvs-lucene compatibility rules)
- #1020
- Add a CI matrix testing supported libcuvs × cuvs-java × cuvs-lucene combinations
API compatibility
- Publish an API compatibility policy for cuvs-java/cuvs-lucene (no guarantee across minors, Lucene-style)
Index file format compatibility
- Publish an index format backward-compatibility guarantee (readable through the next major of lucene)
- Start working on a corpus of indices from older releases and test reading them or add a harness to be able to generate old indices on demand.
- Add backward-compatibility test matrix across Lucene versions × cuvs releases
Cross-platform compatibility
- Add explicit runtime checks + error messages for missing CUDA/insufficient VRAM when opening GPU indices
- Document platform/portability limits of GPU index files (snapshot/restore across clusters)
Performance guarantees
- #2194
- #1856
- #1623
- #2195
- #2428
- #2014
- #2180 Ideally also stand up a benchmark-results dashboard
- Expand benchmarks to run across different GPUs (RTX Pro 6000, H100, A10, L40/L40S) and multiple datasets
Testing
- #2407 Expand with updates and deletes for some records
- Expand reproducible randomized testing with seeds in cuvs-lucene, surface test seeds in CI logs and treat seed-specific failures as real bugs, not flaky
- #2105
[^1]: Items marked with [26.12] are targeted for the [26.12] release.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named. Start by reviewing the linked issues (#2517, #2410, #1682, #1020, and the performance/testing issues), then group the checklist by compatibility, release, performance, and testing work. This tracker is done when its listed cuvs-lucene robustness requirements are completed and each checklist item is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, ci-cd, documentation, performance, search, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100