lance-format / lance-format/lance

bug: load_or_build_quantizer validates its params after sampling, and carries a dead match arm

Open Beginner friendly
#9,367 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

LABEL: bug

Description

IvfIndexBuilder::load_or_build_quantizer in rust/lance/src/index/vector/builder.rs returns early when a quantizer is already present:

if self.quantizer.is_some() {
    return Ok(self.quantizer.clone().unwrap());
}

Everything after that point therefore runs with self.quantizer == None, which leaves two dead things behind and one check in the wrong place:

  • The tail match match &self.quantizer { Some(q) => q.clone(), None => { ... } } can only ever take its None arm.
  • That None arm holds the only validation of quantizer_params, so a builder constructed with neither params nor a quantizer samples the training data and computes residuals first, and only then errors.
  • sample_size_hint's None => 256 * 256 arm is commented "here it must be retrain", but retrain arrives with quantizer: Some(..) and returns at the top, so that default is unreachable too.

None of this is user-visible: the params-missing case is reachable only by calling IvfIndexBuilder::new(.., None, ..) explicitly, and it already ended in the same error, just after the sampling work. It is dead code plus a late check, which rust/CLAUDE.md asks to remove rather than keep.

Expected behavior

Validate the params once, before the sampling and residual work, and drop the arms that cannot be taken.

Lance version

13.0.0-beta.4 (main)

Language binding

Rust

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read rust/lance/src/index/vector/builder.rs, starting at IvfIndexBuilder::load_or_build_quantizer, and trace the parameter validation, sampling, residual work, and early return. Done means missing parameters are checked before sampling and residual computation, while the unreachable quantizer match arm and sample_size_hint default arm are removed; verify the relevant Rust tests or checks pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.