apache / apache/arrow

[R] Empty JSON object parsed as corrupt data frame

Open
#31,244 1 comment 0 reactions 0 assignees View on GitHub
Component: R Status: needs champion Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

If you have a JSON object field that is always empty, it seems to be not handled well, whether or not a schema is provided that tells Arrow what should be in that object.

```r

library(arrow)
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp

json_val <- '{
"rows": [
{"empty": {} },
{"empty": {} },
{"empty": {} }
]
}'
# Remove newlines
json_val <- gsub("\n", "", json_val)

json_file <- tempfile()
writeLines(json_val, json_file)

schema <- schema(field("rows", list_of(struct(empty = struct(y = int32())))))
raw <- read_json_arrow(json_file, schema=schema)
raw$rows$empty
#> Error: Corrupt x: no names
```

**Reporter**: [Will Jones](https://issues.apache.org/jira/browse/ARROW-15803) / @wjones127

**Note**: *This issue was originally created as [ARROW-15803](https://issues.apache.org/jira/browse/ARROW-15803). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with the R reproduction using read_json_arrow() and the supplied schema, then trace how the empty JSON object in the rows field is converted to a data frame. Done means repeated empty objects parse successfully without the “Corrupt x: no names” error, both with and without the schema.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.