apache / apache/parquet-format
Support filtering on group-is-null in filter2 API
- Dominant language
- Thrift
- Stars
- 2.6k
- Forks
- 508
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 4
Description
Currently filters only apply to primitive (leaf node) columns. But given the column "a.b.c.d.e.f" it seems reasonable to want to filter on "a.b.c" != null or something like that.
**Reporter**: [Alex Levenson](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=alexlevenson) / @isnotinvain
**Note**: *This issue was originally created as [PARQUET-43](https://issues.apache.org/jira/browse/PARQUET-43). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
Research direction
Start by locating the filter2 API and its handling of nested column paths. Define the work as allowing a group-is-null condition on an intermediate path such as a.b.c, then verify that the API can filter on that condition rather than only on leaf columns.
Written by the indexing model from the issue text.
Assessment
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100