new feature: expose seekable and positioned streaming reads in Java
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Feature Description
Support Java consumers that need seek, positioned reads, or tail reads without materializing an entire object or implementing their own S3 requests.
### Problem and Solution
A concrete caller is Apache Iceberg S3InputStream: it implements seek, readFully(position,...), and readTail(...). It reuses a current response for small forward seeks and otherwise opens a stream at the desired offset. Its range helpers return InputStreams rather than full-object arrays.
OpenDAL Java exposes createInputStream and whole-range read, but OperatorInputStream extends InputStream without a seek/positioned-read interface. Whole-range byte arrays alone are not equivalent to a reusable seekable stream. The Rust core already has Reader and seek-capable adapters; reuse them where their contract fits.
Acceptance criteria:
- Define cursor behavior for seek and independent positioned reads, including whether positioned reads change the sequential cursor.
- Support bounded memory, EOF/short reads, close, and failure propagation.
- Preserve relevant version/conditional-read guarantees across multiple requests; report gaps instead of silently mixing object versions.
- Avoid requiring an unconditional metadata request when an existing sequential read does not need length.
- Validate through an actual format reader with a recorded query/read trace. Do not invent uniformly spaced ranges and call them an Iceberg workload.
This is an additive capability/design request related to #6226, #7909, and #8251. It does not require taking a dependency on Iceberg or promise query acceleration before measurement.
### Additional Context
[Source reference](https://github.com/apache/iceberg/blob/af86b8399ca794bfe482eae85ba7e79ab0c49adc/aws/src/main/java/org/apache/iceberg/aws/s3/S3InputStream.java#L111). OpenDAL API inventory was checked at `b6cf44f7b8a1523409e0e998e478c996ac970f03`.
Contributor guide
Research direction
Start with Java's OperatorInputStream and the Rust core's Reader and seek-capable adapters, then compare their contracts with Apache Iceberg's S3InputStream reference. Validate the design through an actual format reader and recorded query/read trace. Done means bounded-memory seek and positioned reads with defined cursor, EOF, close, failure, and version-consistency behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, rust
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100