[filesystem] HadoopDataInputStream.seek loops forever past EOF
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
I searched titles, bodies, and comments for `HadoopDataInputStream`, `skipFully`, `seek past EOF`, `HDFS seek hang`, `Hadoop skip EOF`, and related filesystem terms. I also inspected the open Hadoop dependency-alignment PR #3699; it does not touch this code path.
### Fluss version
- `main (development)` at `31621117db9b21e8e00a6e716b3a13b7b66e18be`
- The same implementation is present in `v0.9.1-incubating`, `v0.9.0-incubating`, and `v0.8.0-incubating`
### Please describe the bug 🐞
`HadoopDataInputStream.seek(long)` optimizes small forward seeks by calling `skipFully(delta)`. If the requested position is past EOF, the underlying `InputStream.skip(long)` returns `0`. The loop in `skipFully` then makes no progress and spins forever:
https://github.com/apache/fluss/blob/31621117db9b21e8e00a6e716b3a13b7b66e18be/fluss-filesystems/fluss-fs-hadoop/src/main/java/org/apache/fluss/fs/hdfs/HadoopDataInputStream.java#L132-L135
This contradicts the public `FSDataInputStream.seek` contract, which says callers cannot seek past the end of the stream and reports seek errors through `IOException`:
https://github.com/apache/fluss/blob/31621117db9b21e8e00a6e716b3a13b7b66e18be/fluss-common/src/main/java/org/apache/fluss/fs/FSDataInputStream.java#L36-L43
Java's `InputStream.skip` contract permits returning fewer bytes than requested, including `0`.
#### Reproduction
The existing `SeekableByteArrayInputStream` test fixture already returns `0` from `skip` at EOF. Adding this focused case to `HadoopDataInputStreamTest` reproduces the hang:
```java
@Test
void testSeekPastEndOfStream() {
FSDataInputStream input =
new FSDataInputStream(new SeekableByteArrayInputStream(new byte[1]));
HadoopDataInputStream stream = new HadoopDataInputStream(input);
assertThatThrownBy(() -> stream.seek(2)).isInstanceOf(EOFException.class);
}
```
Run it with:
```bash
./mvnw -pl fluss-filesystems/fluss-fs-hadoop -am \
-Dtest=HadoopDataInputStreamTest \
-Dsurefire.failIfNoSpecifiedTests=false test
```
I also reproduced this twice against Hadoop's real local filesystem implementation using a one-byte file and `seek(2)`. Both runs timed out after two seconds:
```text
before seek target=2 length=1
TIMEOUT after 2s (seek did not return)
```
The nearby control `seek(1)` returned normally with position `1`.
#### Expected behavior
The call should promptly throw `IOException`/`EOFException` because the requested position is past EOF.
#### Actual behavior
`seek(2)` never returns. The thread remains in the `skipFully` loop.
No special configuration or external service is required.
### Solution
Delegate `skipFully` to Hadoop's existing `org.apache.hadoop.io.IOUtils.skipFully(InputStream, long)`, or implement the same zero-progress handling: when `skip` returns `0`, read one byte and throw `EOFException` if EOF has been reached.
A regression test can be added to the existing `HadoopDataInputStreamTest`. This should require no new dependency and no API or storage-format change.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with fluss-filesystems/fluss-fs-hadoop/src/main/java/org/apache/fluss/fs/hdfs/HadoopDataInputStream.java, focusing on seek and skipFully, then read HadoopDataInputStreamTest and the existing SeekableByteArrayInputStream fixture. Run the provided Maven test command and add the seek-past-EOF regression case. Done means seek(2) on a one-byte stream promptly throws EOFException or IOException while the nearby seek(1) behavior remains valid.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100