lance-format / lance-format/lance

bug: PartitionTransformer::new unwraps a fallible index build

Open
#9,348 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

LABEL: bug

Description

PartitionTransformer::new in rust/lance-index/src/vector/ivf/transform.rs builds the centroid lookup index with .unwrap():

let index = SimpleIndex::may_train_index(
    centroids.values().clone(),
    centroids.value_length() as usize,
    distance_type,
)
.unwrap();

may_train_index is genuinely fallible — it calls FixedSizeListArray::try_new_from_values twice and SimpleIndex::try_new, which builds an HNSW over the centroids. rust/CLAUDE.md rules that out: "Never use .unwrap(), .expect(), panic!(), or assert!() in library code for fallible operations".

The fallible part is reached when the HNSW speedup for centroid assignment is in play: under SimpleIndexStatus::Auto that needs at least 1,000,000 centroid values, and LANCE_USE_HNSW_SPEEDUP_INDEXING can force it on regardless. So a large-scale index build is the case that turns a training error into a panic instead of a returned error.

Expected behavior

The error propagates. PartitionTransformer::new and the IvfTransformer constructors that call it return Result, the way new_ivf_transformer_with_quantizer already does.

Lance version

13.0.0-beta.4 (main)

Language binding

Rust

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in rust/lance-index/src/vector/ivf/transform.rs at PartitionTransformer::new and trace the IvfTransformer constructors that call it. Follow may_train_index and its existing Result paths, including the HNSW speedup condition and environment-variable override. Done means index-training failures propagate as errors instead of causing a panic.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.