apache / apache/paimon

[Bug] Java Api Batch Read predicate pushdown filtering does not take effect

Open
#5,915 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

1.2.0

### Compute Engine

JavaAPI

### Minimal reproduce step

Code like this, i try to build a predicate. and i can see log, but it does not take effect.
```
Table table = getPaimonTable(catalogName, databaseName, tableName);

PredicateBuilder builder = new PredicateBuilder(table.rowType());
ReadBuilder readBuilder = table.newReadBuilder()
// .withFilter(buildPredicate(builder, table.rowType(), filterList))
.withFilter(builder.equal(0, PaimonTypeConverter.valueToPaimon("000ffecf2c6f97f311e32b0998d51004", table.rowType().getField("id").type())))
.withProjection(buildProjection(table.rowType(), fieldList));
List splits = readBuilder.newScan().plan().splits();

RecordReader reader = readBuilder.newRead().createReader(splits);
List> rows = new ArrayList<>();
reader.forEachRemaining(internalRow -> {
HashMap row = new HashMap<>();
fieldList.forEach(field -> row.put(field.getFieldName(), PaimonTypeConverter.getRowDataSafe(table.rowType().getField(field.getSourceFieldName()), internalRow)));
rows.add(row);
});
return rows;
```

```
2025-07-17T14:23:30.285+08:00 INFO 36720 --- [nio-8080-exec-5] o.apache.hadoop.fs.s3a.S3AInputStream : Switching to Random IO seek policy
2025-07-17T14:23:30.298+08:00 INFO 36720 --- [nio-8080-exec-5] o.apache.hadoop.fs.s3a.S3AInputStream : Switching to Random IO seek policy
2025-07-17T14:23:30.309+08:00 INFO 36720 --- [-THREAD-POOL-t2] o.apache.hadoop.fs.s3a.S3AInputStream : Switching to Random IO seek policy
2025-07-17T14:23:30.315+08:00 INFO 36720 --- [nio-8080-exec-5] o.a.p.s.o.a.p.f.compat.FilterCompat : Filtering using predicate: eq(id, Binary{"000ffecf2c6f97f311e32b0998d51004"})
2025-07-17T14:23:30.348+08:00 INFO 36720 --- [nio-8080-exec-5] o.apache.hadoop.fs.s3a.S3AInputStream : Switching to Random IO seek policy
2025-07-17T14:23:30.364+08:00 INFO 36720 --- [nio-8080-exec-5] o.apache.hadoop.io.compress.CodecPool : Got brand-new decompressor [.zstd]
```

### What doesn't meet your expectations?

query result dose not filtered

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the Java API read using Table.newReadBuilder(), PredicateBuilder.equal(), withFilter(), and the scan/read path shown in the report. Inspect the predicate log from FilterCompat and verify whether newScan().plan().splits() and createReader(splits) return only rows matching the id; done means the query result is filtered as requested.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.