[IPC] Lower level API to access `requirement_alignment` and other extensions
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 168
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
As part of https://github.com/apache/arrow-rs/pull/5554 @hzuo added the ability to enforce alignment
We could add a `with_require_alignment` to the [`FileDecoder`](https://docs.rs/arrow-ipc/51.0.0/arrow_ipc/reader/struct.FileDecoder.html)
However, given the way the [`read_dictionary`](https://docs.rs/arrow-ipc/51.0.0/arrow_ipc/reader/fn.read_dictionary.html) is structured, there was no way to add this option without breaking the API, as @tustvold noted https://github.com/apache/arrow-rs/pull/5554/files#r1549239776
Thus the require_alignment argument is now not exposed
**Describe the solution you'd like**
It would be nice to have a way to add new options in a way that did not break API compatibility going forward, and expose the `requirement_alignment` feature
**Describe alternatives you've considered**
@tustvold suggests https://github.com/apache/arrow-rs/pull/5554/files#r1550178314
> possibly some sort of `Decoder` struct with a member function on it...
>
> Ideally it would be designed so that it could be shared across things like `StreamDecoder` and `FileDecoder`
**Additional context**
Contributor guide
Research direction
Start by reviewing the FileDecoder API and the read_dictionary function linked in the issue, then read the referenced pull request discussions. Compare how options would need to apply across FileDecoder and StreamDecoder. Done means a compatibility-preserving design exposes requirement_alignment and leaves a clear path for future decoder options.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100