lance-format / lance-format/lance
split sample rate for IVF and SQ training
Open
Nobody has claimed this yet.
A-index
enhancement
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
total sample size is defined as sample_rate * num_centroids or sample_rate * (2 ^ n_bits), for quantization
the orders of magnitude for each is usually very different; the means that using the same param makes no sense.
Maybe change to sample_size?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the IVF and SQ training code and tracing how the current sample_rate parameter determines total sample size. Determine the separate sampling controls needed for each training path, then add coverage showing that IVF and quantization can use independent sample sizes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100