new feature: avoid synchronous lookahead after Java InputStream fills the destination
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Feature Description
Return a completed `OperatorInputStream.read(byte[], off, len)` without fetching the next native chunk first.
### Problem and Solution
After the copy loop consumes the requested length, the method calls `readNextBytes` again if the current chunk is exhausted. The destination is already full, but the caller can still wait for another storage read. This adds avoidable latency when the next chunk is slow.
The next native read could be deferred until the next call actually needs data. Preserve positive byte counts, EOF/error ordering, zero-length reads, and close behavior.
Acceptance criteria:
- Add a deterministic test with a ready first chunk and a blocked second chunk: a destination exactly matching the first chunk must be returned before releasing the second chunk.
- Verify subsequent bytes and EOF, partial reads, zero-length reads, and errors.
- Measure first-consumer-byte latency separately from aggregate throughput.
### Additional Context
[Bulk read and trailing readNextBytes](https://github.com/apache/opendal/blob/b6cf44f7b8a1523409e0e998e478c996ac970f03/bindings/java/src/main/java/org/apache/opendal/OperatorInputStream.java#L61)
This concerns generic Java stream scheduling, not a service-specific S3 failure. It does not require buffering a complete object or changing core transport concurrency.
Source references are pinned to `b6cf44f7b8a1523409e0e998e478c996ac970f03`. This request describes an optimization opportunity; it does not claim a measured end-to-end speedup.
Contributor guide
Research direction
Start with OperatorInputStream.read(byte[], off, len) and readNextBytes in bindings/java/src/main/java/org/apache/opendal/OperatorInputStream.java at the referenced commit. Add a deterministic test using a ready first chunk and blocked second chunk, then verify subsequent bytes, EOF, partial and zero-length reads, errors, close behavior, and separate first-byte latency from aggregate throughput.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100