replace bit-at-a-time null bitmap filtering with word-level
Open
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
When filtering arrays with a null bitmap, the current implementation walks the bitmap one bit at a time, one byte load and branch per row. For mid-selectivity filters (~50% kept) this loop dominates `filter_bits`. we should replace it with a word-level approach that reduces the null bitmap pass from `O(selected_count)` byte loads to `O(filter_len/64)` u64 loads.
Contributor guide
Research direction
Locate filter_bits and trace how selected rows and the null bitmap are represented in its callers. Read the current bit-at-a-time loop first; done means preserving filtering behavior while reducing the null-bitmap pass to word-level u64 loads with O(filter_len/64) work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100