lance-format / lance-format/lance
Evaluate primitive predicates directly on encoded pages
Open
@Xuanwo is already working on this.
Since Aug 21, 2026.
A-encoding
enhancement
performance
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Goal
Evaluate supported primitive predicates directly on encoded pages when doing so is cheaper than decoding the predicate column to Arrow.
Context
Implementation and benchmark: https://github.com/lance-format/lance/pull/8661
Contract
- Support exact i32/u32 comparisons on Constant, RLE, and bitpacked pages.
- Select the encoded path independently per fragment from encoding, sampled selectivity, selected ranges, and predicted read amplification.
- Retain the ordinary decoded filter for unsupported or unfavorable cases.
- Stop encoded evaluation when bounded amplification or selected-range limits are exceeded.
- Short-circuit Constant outcomes only when metadata and nullability make the result exact.
- Preserve query results and fallback behavior for every unsupported encoding and predicate.
Completion gates
- https://github.com/lance-format/lance/pull/8661 is reviewed and merged.
- Exact-result and fallback coverage passes across supported encodings, nullability, and fragmented inputs.
- Representative Local FS and object-storage measurements document accepted wins and rejected-path overhead.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.