apache / apache/iceberg

EOF not handled correctly in single byte read in cloud storage InputStream implementations

Open
#16,062 3 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.10.1 (latest release)

### Query engine

None

### Please describe the bug 🐞

### Description

#16055 identified and fixed an EOF handling bug in `GCSInputStream`. But it looks like the same bug exists in all other cloud storage `InputStream` implementations.

**Impact:** The single-byte `read()` bug can cause an infinite loop for callers that reads until EOF. In practice, Iceberg typically reads files using range reads at known offsets rather than sequential reads to EOF, so this is unlikely to be hit in the hot path.

There is also a metric bug in the buffered read called out in the PR.

Affected implementations:
- [`S3InputStream#read()`](https://github.com/apache/iceberg/blob/f984c28b215c56846c632ccc5a368f4f4afe0b5d/aws/src/main/java/org/apache/iceberg/aws/s3/S3InputStream.java#L124-L125)
- [`ADLSInputStream#read()`](https://github.com/apache/iceberg/blob/f984c28b215c56846c632ccc5a368f4f4afe0b5d/azure/src/main/java/org/apache/iceberg/azure/adlsv2/ADLSInputStream.java#L117-L118)
- `OSSInputStream` (aliyun)
- `EcsSeekableInputStream` (dell)

GCS fix from @vladislav-sidorovich : https://github.com/apache/iceberg/pull/16055

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.