lance-format / lance-format/lance
bug: IVF_RQ with the hamming metric panics in the kmeans trainer
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Description
Building an IVF_RQ index with metric_type = Hamming panics:
thread '...' panicked at rust/lance-index/src/vector/kmeans.rs:553:21:
KMeans::find_partitions: hamming is not supported
Nothing rejects the combination first. build_vector_index_impl in rust/lance/src/index/vector.rs checks the element type for the flat and PQ arms but not the metric, RabitQuantizer::build ignores the distance type entirely (its signature takes it as _), and the RQ transform's own "distance type not supported" error sits downstream of IVF training. So the build gets as far as assigning partitions and then panics inside kmeans.
The combination cannot be made to work: RaBitQ encodes residuals under L2 or dot, and Hamming is a binary-vector metric for UInt8 columns while RQ requires float vectors.
Expected behavior
Reject IVF_RQ with Hamming as invalid input before any training, the way the other unsupported combinations are rejected.
Lance version
13.0.0-beta.4 (main)
Language binding
Rust
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in rust/lance/src/index/vector.rs at build_vector_index_impl and compare its validation with the flat and PQ arms. Read RabitQuantizer::build and the IVF training path in rust/lance-index/src/vector/kmeans.rs; done means IVF_RQ with Hamming is rejected before training instead of panicking.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100