apache / apache/parquet-format

Support filtering on group-is-null in filter2 API

Open
#261 0 comments 0 reactions 0 assignees View on GitHub
Priority: Minor Type: enhancement
Dominant language
Thrift
Stars
2.6k
Forks
508
Avg merge
2d 8h
Merged PRs (30d)
4

Description

Currently filters only apply to primitive (leaf node) columns. But given the column "a.b.c.d.e.f" it seems reasonable to want to filter on "a.b.c" != null or something like that.

**Reporter**: [Alex Levenson](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=alexlevenson) / @isnotinvain

**Note**: *This issue was originally created as [PARQUET-43](https://issues.apache.org/jira/browse/PARQUET-43). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by locating the filter2 API and its handling of nested column paths. Define the work as allowing a group-is-null condition on an intermediate path such as a.b.c, then verify that the API can filter on that condition rather than only on leaf columns.

Written by the indexing model from the issue text.

Assessment

Domain
api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.