Add string coercion when decoding json
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
When Spark decodes JSON data where values don't match the expected schema type, it silently coerces incompatible values to strings instead of failing or preserving the original structure.
When decoding the following JSON with a schema of `map`: `{"hello": [1,2,3]}`, Spark automatically converts the array value to its string representation, resulting in `{"hello": "[1,2,3]"}`
Contributor guide
Research direction
No file or test is named. First reproduce the JSON example with the map schema and trace the Rust JSON decoding path that performs the coercion; before coding, clarify whether incompatible values should fail or preserve their original structure, then add coverage for the agreed behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100