apache / apache/arrow-rs

[arrow-avro] Move the writer-wire planning fully into `codec.rs`

Open
#9,651 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 14h
Merged PRs (30d)
167

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

Improve the boundary between the `arrow-avro` codec and decoder by ensuring planning logic related to the skipper is handled in `codec.rs` and not during decoder construction or parsing.

**Describe the solution you'd like**

I'd recommend pushing the writer-wire planning fully into `codec.rs`, instead of carrying full `AvroDataType`s down into `record.rs` and having `Skipper::from_avro` reconstruct the skip tree there. However this can always be done in a follow-up as well.

_Originally posted by @jecsand838 in https://github.com/apache/arrow-rs/issues/9605#issuecomment-4172638199_

**Describe alternatives you've considered**

1. Leaving the code structure as is, which is fine for now, but overtime will likely reduce maintainability.

**Additional context**

N/A

Contributor guide

Open the contributing guide

Research direction

Start by reading arrow-avro's codec.rs and record.rs, then trace Skipper::from_avro to understand where writer-wire planning is currently reconstructed. Done means planning related to the skipper is handled in codec.rs rather than during decoder construction or parsing, without carrying full AvroDataTypes into record.rs.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.