lance-format / lance-format/lance
bug: row_range panics where partition_size returns 0 for an absent partition
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
IvfModel::row_range and IvfModel::partition_size answer the same question about the same input and disagree: for a partition the model does not carry, one panics and the other returns 0.
pub fn row_range(&self, partition: usize) -> Range<usize> {
let start = self.offsets[partition]; // storage.rs:144, panics
let end = start + self.lengths[partition] as usize;
start..end
}
pub fn partition_size(&self, part: usize) -> usize {
self.lengths.get(part).copied().unwrap_or_default() as usize // storage.rs:94, returns 0
}
num_partitions() adds to the confusion: it prefers centroids.len() and only falls back to offsets.len() (storage.rs:86-91), so the count a caller iterates over is not necessarily the one row_range can index.
I could not construct the panic through the public API, and this is worth saying plainly rather than filing it as a live bug. User-supplied partition ids reach load_partition, which rejects out-of-range ids with a clean error, and search-path ids come from find_partitions over the centroids so they are below nlist. The three callers that index without a guard would need a written model whose num_partitions() exceeds offsets.len(), and neither the builder nor the distributed merger can emit one.
The one seam I could not rule out: nothing checks the centroid count against lengths.len() on write or on load, and rust/lance/src/index/vector/ivf.rs:2690 grafts centroids from a different file than the one supplying the lengths, so a mismatched shard set there would reach the panic. I could not produce that from a normal write path.
So this is a contract inconsistency between two neighbouring methods rather than a reachable failure.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with IvfModel::row_range, partition_size, and num_partitions in storage.rs:86-91, 94, and 144, then inspect the centroid grafting path at rust/lance/src/index/vector/ivf.rs:2690. Determine the intended contract for absent or mismatched partitions and add coverage for the relevant model shape; done means the neighboring methods behave consistently without an unhandled panic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100