[C++] Can't use union in read_json
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the usage question you have. Please include as many useful details as possible.
I'm trying to use pyarrow to read a json file into a pyarrow data set.
I am having trouble with the union type though.
applicants.large.jsonl
```
{"Type":"insert","Key":[2],"SeqNo":2,"Timestamp":2,"Fields":[{"Name":"added_id","Value":3},{"Name":"row_key","Value":"/XX"},{"Name":"column_key","Value":"syz"},{"Name":"ref_key","Value":2},{"Name":"updated","Value":"2010-12-08T02:15:07.664543Z"}]}
```
For reference, this is the formatted JSON:
```
{
"Type": "insert",
"Key": [
2
],
"SeqNo": 2,
"Timestamp": 2,
"Fields": [
{
"Name": "added_id",
"Value": 3
},
{
"Name": "row_key",
"Value": "/XX"
},
{
"Name": "column_key",
"Value": "syz"
},
{
"Name": "ref_key",
"Value": 2
},
{
"Name": "updated",
"Value": "2010-12-08T02:15:07.664543Z"
}
]
}
```
Code:
```
data = "applicants.large.jsonl"
schema = pa.schema([
('Type', pa.string()),
('Fields', pa.list_(pa.struct([
('Name', pa.string()),
('Value', pa.union([
pa.field("int64", pa.int64()),
pa.field("string", pa.string()),
], mode='sparse'))
])))
])
table = pa.json.read_json(data, parse_options=pa.json.ParseOptions(explicit_schema=schema))
```
Error:
```
ArrowNotImplementedError: JSON conversion to sparse_union is not supported
```
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.