Use optimized implementation of bloom filter
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
Hey!
I implemented https://github.com/ozgrakkurt/sbbf-rs.
It is an implementation of parquet bloom filters, I checked it against the implementation at `parquet2` and it produces same output.
I would like to integrate it here if it makes sense to do.
A big problem with it may be that it has different implementations including a `aarch64::neon` which wouldn't get tested on GitHub CI.
Contributor guide
Research direction
Start by reviewing the existing Parquet bloom-filter implementation in arrow-rs and compare it with the proposed sbbf-rs implementation and its parquet2 output. Assess the aarch64::neon testing concern; done means the project has a decided integration path with appropriate platform coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100