Throw Exception when predicate contains columns not specified in prejection, to prevent filtering out data improperly
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
As is reported by Parquet-425, data will be filtered out improperly under certain cases.
Before Parquet-425 is fixed, let's throw an Exception to warn the upper application that work-arounds should be done.
**Reporter**: [Liwei Lin(Inactive)](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=proflin) / @lw-lin
**Assignee**: [Liwei Lin(Inactive)](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=proflin) / @lw-lin
#### Externally tracked issue: [https://issues.apache.org/jira/browse/PARQUET-425](https://issues.apache.org/jira/browse/PARQUET-425)
**Note**: *This issue was originally created as [PARQUET-426](https://issues.apache.org/jira/browse/PARQUET-426). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the Parquet-425 report and the related PARQUET-426 issue history to identify the affected predicate and projection handling. Trace where predicates are evaluated against projected columns, then verify that cases with unspecified predicate columns raise an exception rather than filtering data; the issue provides no file or test path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100