lance-format / lance-format/lance

bug: IVF_RQ with the hamming metric panics in the kmeans trainer

Open Beginner friendly
#9,422 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Description

Building an IVF_RQ index with metric_type = Hamming panics:

thread '...' panicked at rust/lance-index/src/vector/kmeans.rs:553:21:
KMeans::find_partitions: hamming is not supported

Nothing rejects the combination first. build_vector_index_impl in rust/lance/src/index/vector.rs checks the element type for the flat and PQ arms but not the metric, RabitQuantizer::build ignores the distance type entirely (its signature takes it as _), and the RQ transform's own "distance type not supported" error sits downstream of IVF training. So the build gets as far as assigning partitions and then panics inside kmeans.

The combination cannot be made to work: RaBitQ encodes residuals under L2 or dot, and Hamming is a binary-vector metric for UInt8 columns while RQ requires float vectors.

Expected behavior

Reject IVF_RQ with Hamming as invalid input before any training, the way the other unsupported combinations are rejected.

Lance version

13.0.0-beta.4 (main)

Language binding

Rust

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in rust/lance/src/index/vector.rs at build_vector_index_impl and compare its validation with the flat and PQ arms. Read RabitQuantizer::build and the IVF training path in rust/lance-index/src/vector/kmeans.rs; done means IVF_RQ with Hamming is rejected before training instead of panicking.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.