apache / apache/paimon

[Bug] RowFormatReaderFactory leaks the input stream when reading the footer fails

Open Beginner friendly
#9,006 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

master, 142f8239b

### Compute Engine

Engine independent.

### Minimal reproduce step

Found by code inspection rather than from a failing job. Read a row format file whose tail is truncated, or whose footer is corrupt, so that footer or block index parsing throws.

### What doesn't meet your expectations?

`RowFormatReaderFactory.createReader` opens the stream first and only transfers ownership to `RowFormatReader` on the last line:

```java
SeekableInputStream in = fileIO.newInputStream(path);

int tailSize = (int) Math.min(TAIL_PREFETCH_SIZE, fileSize);
long tailOffset = fileSize - tailSize;
in.seek(tailOffset);
byte[] tailBuf = new byte[tailSize];
IOUtils.readFully(in, tailBuf);

RowFileFooter footer = RowFileFooter.readFrom(tailBuf, tailSize - RowFileFooter.FOOTER_SIZE);

RowBlockIndex blockIndex;
...
blockIndex = RowBlockIndex.readFrom(in, footer.indexOffset, footer.indexLength);

return new RowFormatReader(in, path, footer, blockIndex, rowType, projection, context.selection());
```

There is no `try` / `catch` anywhere in the method, so a throw from `in.seek`, `IOUtils.readFully`, `RowFileFooter.readFrom` or `RowBlockIndex.readFrom` leaves `in` open with no owner. Scanning a set of files where several are corrupt leaks one stream per file.

### Anything else?

Same interaction as the sibling report on `CachingSeekableInputStream`: with the lifetime tracking added in #8962, a stream that is never closed holds its lease forever, so that entry's `FileIO` is never released. That matches today's behaviour rather than regressing it, but it does keep the fix from reaching the affected entries.

Fix shape: close `in` before rethrowing.

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at RowFormatReaderFactory.createReader and trace ownership of the stream after opening it, especially the footer and block-index parsing paths. Use the truncated-tail or corrupt-footer scenario described in the issue; done means failures close the input stream before being rethrown, including errors from seek, read, footer parsing, and block-index parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.