apache / apache/arrow-rs

Use optimized implementation of bloom filter

Open
#4,213 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 14h
Merged PRs (30d)
167

Description

Hey!

I implemented https://github.com/ozgrakkurt/sbbf-rs.

It is an implementation of parquet bloom filters, I checked it against the implementation at `parquet2` and it produces same output.

I would like to integrate it here if it makes sense to do.

A big problem with it may be that it has different implementations including a `aarch64::neon` which wouldn't get tested on GitHub CI.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing Parquet bloom-filter implementation in arrow-rs and compare it with the proposed sbbf-rs implementation and its parquet2 output. Assess the aarch64::neon testing concern; done means the project has a decided integration path with appropriate platform coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.