Improve performance when filters in RowGroupFilter can match exactly
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
If we can accurately judge by the minMax status, we don’t need to load the dictionary from filesystem and compare one by one anymore.
Similarly , Bloomfilter needs to load from filesystem, it may costs time and memory. If we can exactly determine the existence/nonexistence of the value from minMax or dictionary filters , then we can avoid using Bloomfilter to Improve performance.
For example,
1. read data greater than `x1` in the block, if minMax in status is all greater than `{}x1{`}, then we don't need to read dictionary and compare one by one.
1. If we already have page dictionaries and have compared one by one, we don't need to read BloomFilter and compare.
**Reporter**: [Mars](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=miracle) / @yabola
**Assignee**: [Mars](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=miracle) / @yabola
**Note**: *This issue was originally created as [PARQUET-2237](https://issues.apache.org/jira/browse/PARQUET-2237). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.