apache / apache/arrow-rs

Add string coercion when decoding json

Open
#7,453 7 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 14h
Merged PRs (30d)
167

Description

When Spark decodes JSON data where values don't match the expected schema type, it silently coerces incompatible values to strings instead of failing or preserving the original structure.

When decoding the following JSON with a schema of `map`: `{"hello": [1,2,3]}`, Spark automatically converts the array value to its string representation, resulting in `{"hello": "[1,2,3]"}`

Contributor guide

Open the contributing guide

Research direction

No file or test is named. First reproduce the JSON example with the map schema and trace the Rust JSON decoding path that performs the coercion; before coding, clarify whether incompatible values should fail or preserve their original structure, then add coverage for the agreed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.