[Parquet] Support version in ParquetObjectReader
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
I want to use `ParquetObjectReader` https://github.com/apache/arrow-rs/blob/2273cdc3e16e24f2379b3c96232667cb58881058/parquet/src/arrow/async_reader/store.rs#L258 to read an S3 object with a specific version. This is not possible with the ParquetObjectReader implementation today.
**Describe the solution you'd like**
In `ParquetObjectReader`, add a new `version` field and a new `with_version` method. Change the `AsyncFileReader` and `MetadataSuffixFetch` implementation to use `get_opts` with version if version is specified.
**Describe alternatives you've considered**
Alternative is to maintain a `ParquetObjectReader` implementation that supports object version in our implementation.
**Additional context**
Contributor guide
Research direction
Start in parquet/src/arrow/async_reader/store.rs at ParquetObjectReader and inspect its AsyncFileReader and MetadataSuffixFetch implementations. Trace how object requests are made, then verify that an optional version is passed through get_opts when configured. Done means ParquetObjectReader exposes version configuration and reads the requested S3 object version.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100