lance-format / lance-format/lance
bug: binary_quantization appends a spurious byte when the length is a multiple of 8
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
binary_quantization packs the sign bits of a float slice into bytes and appends the tail byte unconditionally (rust/lance-index/src/vector/bq.rs:69):
let iter = data.chunks_exact(8);
iter.clone()
.map(|c| { ... })
.chain(once(0).map(move |_| {
let mut bits: u8 = 0;
iter.remainder().iter().enumerate().for_each(...);
bits
}))
When data.len() is a multiple of 8 the remainder is empty, so the chained byte is always zero and the output is floor(len / 8) + 1 bytes instead of ceil(len / 8).
The public entry point BinaryQuantization::transform flattens every row through this helper into one UInt8Array, so the per-row width a caller has to assume, ceil(dim / 8), disagrees with what it gets. For a dimension that is a multiple of 8, every row carries one extra zero byte.
Reproduced on 36bd4e27a: 16 values give [85, 85, 0] where [85, 85] is correct, and transform over 3 rows of dimension 8 returns 6 bytes instead of 3.
BinaryQuantization has no callers in this repository, so this is latent rather than a live failure, and RabitQuantizer is what the IVF_RQ path uses. Filing it because the type is public API (lance_index::vector::bq::BinaryQuantization) and the helper predates RaBitQ (#1988).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in rust/lance-index/src/vector/bq.rs at the binary_quantization helper and trace how BinaryQuantization::transform flattens rows. Verify the 16-value example and the three-row dimension-8 case, then ensure dimensions divisible by 8 produce ceil(dim / 8) bytes with no extra tail byte.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100