apache / apache/parquet-format

N-gram Bloom Filter Support

Open
#490 3 comments 0 reactions 0 assignees View on GitHub
Type: enhancement
Dominant language
Thrift
Stars
2.6k
Forks
508
Avg merge
2d 8h
Merged PRs (30d)
4

Description

### Describe the enhancement requested

Some database support bloom filters on n-grams from a String. This facilitates some operations like "like" operations.
Searching for a particular discriminant token with rare n-grams can be greatly sped up.

https://clickhouse.com/docs/optimize/skipping-indexes#bloom-filter-types

see `ngrambf_v1`

Contributor guide

Open the contributing guide

Research direction

Start with the ClickHouse skipping-indexes documentation linked in the issue and the `ngrambf_v1` reference. Determine how n-gram Bloom filter support should be represented in the Parquet format and what compatibility requirements apply. Done means the format change is specified clearly enough to support the described n-gram filtering operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.