Unable to create bloomfilter when writing duplicates values for a field
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
I'm unable to create a bloomfilter for a field, when I perform writes with repeating values. The bloomfilter returned is null when I try to read such a parquet file. If there are no repeating values, the bloomfilter is created without any issue.
The working and non-working case in captured in the below repo
**Reporter**: [Mahesh Pai](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=maheshgpai)
**Note**: *This issue was originally created as [PARQUET-2484](https://issues.apache.org/jira/browse/PARQUET-2484). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the working and failing cases from the linked parquet-mr-test repository, starting at src/test/java/com/mahesh/test/AppTest.java around line 73. Compare bloomfilter creation and reading for repeated versus unique field values, then add a regression test showing that the bloomfilter is non-null for duplicate values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100