apache / apache/arrow-rs

Expose arrow_json::writer::encoder::make_encoder_impl() for re-use in custom encoders

Open
#5,733 7 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 14h
Merged PRs (30d)
167

Description

In our project we have a set of JSON serialization rules not covered by `arrow_json`.

For example, there are binary columns that already contain a valid JSON that needs to be just embedded "as is" into the output stream. We also have cases when we'd like to serialize some StructArrays in non-canonical way without any prior transformation.

Public [make_encoder_impl()](https://github.com/apache/arrow-rs/blob/119db5cdc64d52955a24fc08888f10db69019062/arrow-json/src/writer/encoder.rs#L53) would allow us to re-use existing arrow encoders in our serializers.

Contributor guide

Open the contributing guide

Research direction

Start in arrow-json/src/writer/encoder.rs at make_encoder_impl() and inspect how the existing encoder construction is exposed. Confirm the API needed by custom serializers and ensure the function can be reused outside its current scope without changing existing encoder behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.