lance-format / lance-format/lance

bug: IVF_SQ and IVF_RQ on a UInt8 column train the whole IVF model before rejecting the dtype

Open Beginner friendly
#9,415 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Description

build_vector_index_impl in rust/lance/src/index/vector.rs checks the vector column's element type up front for the flat and PQ index types, with an arm that returns Build Vector Index: invalid data type: .... The IvfSq and IvfRq arms do not check it: they go straight to the builder, and the dtype is rejected much later by the quantizer's own build ("SQ builder: unsupported data type", "Unsupported data type").

Where that costs something is Hamming. With a UInt8 column and metric_type = Hamming, kmeans accepts the dtype, so the build samples the training data and trains the entire IVF model before the SQ quantizer refuses it. With L2 the failure lands earlier, in train_ivf_model, but still after the training sample is read, and the message talks about the metric rather than the index type's requirement.

Expected behavior

Reject a non-float element type for SQ and RQ where the other index types are checked, before any data is read.

Lance version

13.0.0-beta.4 (main)

Language binding

Rust

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in rust/lance/src/index/vector.rs at build_vector_index_impl and compare the existing flat and PQ element-type checks with the IvfSq and IvfRq arms. Add the equivalent early rejection for non-float types, then verify that UInt8 columns are rejected before training data is read and that the error identifies the invalid data type.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.