lance-format / lance-format/lance
Add fused batch distance kernels for vector search
Open
@BubbleCal is already working on this.
Since May 18, 2026.
A-index
enhancement
performance
rust
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Parent: lance-format/lance#6820
Related: lance-format/lance#6821
Linear: lance-format/lance#6823
Linear parent: lance-format/lance#6820
Goal
Add low-level distance kernels that accept a batch of query vectors and compute distances against a data batch in one fused operation.
Batch query should not only mean shared scan/planning. The hot distance path should avoid a loop that slices one query vector at a time and calls the existing single-query Arrow distance function repeatedly.
Scope
- Add a kernel API in the vector distance / linalg layer that accepts multiple query vectors plus one data vector batch.
- Return distances in a layout that downstream top-k code can consume without excessive reshaping or allocation.
- Cover common dense metrics such as L2, cosine, and dot where applicable.
- Integrate the fused kernel into flat KNN batch query execution.
- Keep scalar fallback paths for unsupported types or platforms.
Acceptance criteria
- Unit tests cover output ordering/layout for multiple queries and multiple rows.
- Flat KNN batch query uses the fused kernel where supported instead of per-query kernel loops.
- Benchmarks compare repeated single-query kernels, current batch query behavior, and fused batch kernels.
- The benchmark report includes dimensions, query batch size, row count, CPU target features, and p50/p95/p99 latency.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.