lance-format / lance-format/lance

bug: IVF_HNSW_PQ recall assertions fail intermittently because k-means init is unseeded

Open
#8,764 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug performance
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

test_create_ivf_hnsw_pq_4bit and test_create_ivf_hnsw_pq_multivec fail their
assert_ge!(recall, 0.5) check intermittently, on main as well as on PRs.

Observed in the last day:

where test recall
main 91aee6f91, linux-build test_create_ivf_hnsw_pq_4bit::case_3_dot 0.4
PR #8431, mac-build (stable) test_create_ivf_hnsw_pq_4bit::case_3_dot 0.4
PR #8755, linux-arm test_create_ivf_hnsw_pq_multivec 0.3

Each run was otherwise green (e.g. 3146 passed, 1 failed), and the failing test
differs by runner, so this is not a regression from any one change.

The input is already deterministic: make_seeded_vector_batch
(rust/lance/src/index/vector/ivf/v2.rs:2615) pins Seed::from(42). The variance
comes from index training instead — rust/lance-index/src/vector/kmeans.rs:930
reads

// TODO: use seed for Rng.
let mut rng = SmallRng::from_os_rng();

so centroid initialization differs every run. With K = 10 the recall metric
moves in steps of 0.1 and the threshold is 5 of 10 neighbours, so a slightly
worse set of centroids flips the assertion.

Threading a seed through KMeansParams for the test path would make these
deterministic, which the existing TODO already anticipates. Lowering the
threshold would also stop the failures but would weaken the check.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with rust/lance-index/src/vector/kmeans.rs:930 and inspect how KMeansParams reaches training for test_create_ivf_hnsw_pq_4bit and test_create_ivf_hnsw_pq_multivec. Compare this with the seeded input setup in rust/lance/src/index/vector/ivf/v2.rs:2615, then run both named tests repeatedly. Done means centroid initialization is deterministic for these tests and the existing recall assertions pass without lowering their threshold.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.