Speed up `BitIndexIterator` by unrolling
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
`BitIndexIterator` is critical to performance of lots of kernels (filter / take and anything dealing with nulls).
This can be made faster
**Describe the solution you'd like**
Implement optimization described here:
https://lemire.me/blog/2019/05/03/really-fast-bitset-decoding-for-average-densities/
**Describe alternatives you've considered**
**Additional context**
Contributor guide
Research direction
Start by locating `BitIndexIterator` in the arrow-rs source and read the linked Lemire article on fast bitset decoding. Compare the current iterator with the described unrolling approach, then verify that the change improves performance for the kernels that depend on it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100