Parquet: ParquetFilters.convert() does not handle AlwaysFalse, may produce incorrect filter pushdown
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Feature Request / Improvement
### Apache Iceberg version
main (unreleased)
### Query engine
Any (Spark, Flink, etc.)
### Problem
`ParquetFilters.convert()` has a TODO on line 45 acknowledging that `AlwaysFalse.INSTANCE` is not handled:
```java
// TODO: handle AlwaysFalse.INSTANCE
if (pred != null && pred != AlwaysTrue.INSTANCE) {
return FilterCompat.get(pred);
} else {
return FilterCompat.NOOP;
}
```
When an Iceberg expression evaluates to `AlwaysFalse`, the visitor returns `AlwaysFalse.INSTANCE` — a custom `FilterPredicate` sentinel. Because it is non-null and not `AlwaysTrue.INSTANCE`, the condition passes and it gets wrapped via `FilterCompat.get(pred)`. Parquet's internal filter infrastructure has no knowledge of Iceberg's `AlwaysFalse` type, so the resulting filter behavior is undefined — it may read all rows instead of none.
### Expected behavior
When the expression evaluates to `AlwaysFalse`, the convert method should either:
- Return a filter that rejects all row groups (e.g., via a tautologically false Parquet predicate), or
- Return `FilterCompat.NOOP` and let Iceberg's own row-group filtering handle the exclusion
### Proposed fix
Add `AlwaysFalse.INSTANCE` to the condition:
```java
if (pred != null && pred != AlwaysTrue.INSTANCE && pred != AlwaysFalse.INSTANCE) {
return FilterCompat.get(pred);
} else if (pred == AlwaysFalse.INSTANCE) {
// construct a false predicate or signal no rows
} else {
return FilterCompat.NOOP;
}
```
### Affected file
ParquetFilters.java (line 45–50)
### Query engine
None
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
Contributor guide
Research direction
Start with ParquetFilters.java around lines 45–50 and trace how convert handles AlwaysTrue.INSTANCE, AlwaysFalse.INSTANCE, and FilterCompat.NOOP. Determine which supported behavior matches the issue’s expected outcomes, then verify that AlwaysFalse no longer produces an undefined Parquet filter while normal predicates remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100