lance-format / lance-format/lance

Add fused batch distance kernels for vector search

Open
#6,823 0 comments 0 reactions 1 assignee View on GitHub

@BubbleCal is already working on this.

Since May 18, 2026.

A-index enhancement performance rust
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Parent: lance-format/lance#6820
Related: lance-format/lance#6821
Linear: lance-format/lance#6823
Linear parent: lance-format/lance#6820

Goal

Add low-level distance kernels that accept a batch of query vectors and compute distances against a data batch in one fused operation.

Batch query should not only mean shared scan/planning. The hot distance path should avoid a loop that slices one query vector at a time and calls the existing single-query Arrow distance function repeatedly.

Scope

  • Add a kernel API in the vector distance / linalg layer that accepts multiple query vectors plus one data vector batch.
  • Return distances in a layout that downstream top-k code can consume without excessive reshaping or allocation.
  • Cover common dense metrics such as L2, cosine, and dot where applicable.
  • Integrate the fused kernel into flat KNN batch query execution.
  • Keep scalar fallback paths for unsupported types or platforms.

Acceptance criteria

  • Unit tests cover output ordering/layout for multiple queries and multiple rows.
  • Flat KNN batch query uses the fused kernel where supported instead of per-query kernel loops.
  • Benchmarks compare repeated single-query kernels, current batch query behavior, and fused batch kernels.
  • The benchmark report includes dimensions, query batch size, row count, CPU target features, and p50/p95/p99 latency.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.