GPU search hardcodes query batch size 1 — no multi-query batching
@imotov is already working on this.
Since Aug 17, 2026.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Summary
cuvs-lucene always passes a 1 × dim query matrix to cuVS. CagraSearchParams.maxQueries is never set. cuVS supports batched search natively; the Lucene layer does not expose it.
Reproducer
From a cuvs-lucene checkout:
rg 'deviceBuilder\([^,]+, 1,' src/main/java/com/nvidia/cuvs/lucene/
Both GPU search paths hardcode batch size 1:
GPUKnnFloatVectorQuery.java(~line 195) — multi-segment pathCuVS2510GPUVectorsReader.java(~line 458) — per-segment fallback
Neither calls CagraSearchParams.Builder.withMaxQueries(n) for n > 1.
Runtime check: run any GPUKnnFloatVectorQuery search; one cuVS cagraSearch call is issued per Lucene query. MultiPartitionCagraSearch batches segments, not query vectors — still one query per call.
Expected vs actual
- Expected: option to search N query vectors in one GPU call (as cuVS Java API allows via N-row
CuVSMatrix+maxQueries). - Actual: one query vector per search; throughput scaling in benchmarks uses
queryThreads(N concurrent batch-1 searches).
Workaround
Use multiple search threads (queryThreads > 1). Not equivalent to in-kernel query batching.
Environment
cuvs-lucene 25.10+, CuVS2510GPUSearchCodec, GPUKnnFloatVectorQuery.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.