apache / apache/iceberg

Parquet: ParquetFilters.convert() does not handle AlwaysFalse, may produce incorrect filter pushdown

Open
#16,032 2 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

### Apache Iceberg version

main (unreleased)

### Query engine

Any (Spark, Flink, etc.)

### Problem

`ParquetFilters.convert()` has a TODO on line 45 acknowledging that `AlwaysFalse.INSTANCE` is not handled:

```java
// TODO: handle AlwaysFalse.INSTANCE
if (pred != null && pred != AlwaysTrue.INSTANCE) {
return FilterCompat.get(pred);
} else {
return FilterCompat.NOOP;
}
```

When an Iceberg expression evaluates to `AlwaysFalse`, the visitor returns `AlwaysFalse.INSTANCE` — a custom `FilterPredicate` sentinel. Because it is non-null and not `AlwaysTrue.INSTANCE`, the condition passes and it gets wrapped via `FilterCompat.get(pred)`. Parquet's internal filter infrastructure has no knowledge of Iceberg's `AlwaysFalse` type, so the resulting filter behavior is undefined — it may read all rows instead of none.

### Expected behavior

When the expression evaluates to `AlwaysFalse`, the convert method should either:
- Return a filter that rejects all row groups (e.g., via a tautologically false Parquet predicate), or
- Return `FilterCompat.NOOP` and let Iceberg's own row-group filtering handle the exclusion

### Proposed fix

Add `AlwaysFalse.INSTANCE` to the condition:

```java
if (pred != null && pred != AlwaysTrue.INSTANCE && pred != AlwaysFalse.INSTANCE) {
return FilterCompat.get(pred);
} else if (pred == AlwaysFalse.INSTANCE) {
// construct a false predicate or signal no rows
} else {
return FilterCompat.NOOP;
}
```

### Affected file

ParquetFilters.java (line 45–50)

### Query engine

None

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Research direction

Start with ParquetFilters.java around lines 45–50 and trace how convert handles AlwaysTrue.INSTANCE, AlwaysFalse.INSTANCE, and FilterCompat.NOOP. Determine which supported behavior matches the issue’s expected outcomes, then verify that AlwaysFalse no longer produces an undefined Parquet filter while normal predicates remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.