apache / apache/fluss

[Bug][Lake][Paimon] Fix the situation where resources may not be free up when reading paimon

Open Beginner friendly
#2,790 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Fluss version

0.9.0 (latest release)

### Please describe the bug 🐞

Issue:

In the constructor, after tableRead.createReader() creates a RecordReader and converts it to a CloseableIterator via toCloseableIterator(), if the subsequent
PaimonRowAsFlussRecordIterator construction throws an exception, the CloseableIterator resource is never closed, causing a resource leak.

```
org.apache.paimon.reader.RecordReader recordReader =
tableRead.createReader(split.dataSplit());
iterator =
new PaimonRecordReader.PaimonRowAsFlussRecordIterator(
recordReader.toCloseableIterator(), paimonRowType);
```

Root Cause:

The toCloseableIterator() call returns a resource that must be closed. If an exception occurs during the PaimonRowAsFlussRecordIterator construction, the constructor fails before
the iterator is assigned to the field, leaving no way for the caller to close the leaked resource.

### Solution

Extract the CloseableIterator to a local variable and wrap the construction in a try-catch block to ensure proper cleanup on failure:

```
org.apache.paimon.reader.RecordReader recordReader =
tableRead.createReader(split.dataSplit());
org.apache.paimon.utils.CloseableIterator closeableIterator =
recordReader.toCloseableIterator();
try {
iterator =
new PaimonRecordReader.PaimonRowAsFlussRecordIterator(
closeableIterator, paimonRowType);
} catch (Throwable t) {
try {
closeableIterator.close();
} catch (Exception closeException) {
t.addSuppressed(closeException);
}
throw t;
}
```

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the PaimonRecordReader.PaimonRowAsFlussRecordIterator construction after RecordReader.toCloseableIterator() is called. Verify that a construction failure closes the returned CloseableIterator and preserves any close exception as suppressed; done means no resource remains open when initialization fails.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.