[Python] read_json error message not clear when duplicate field in schema
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
I had mistakenly put a duplicate field in my schema when calling `pyarrow.json.read_json` with `ParseOptions.explicit_schema`
I was either getting seg faults or an error message that wasn't clear.
Given the nature of json, there shouldn't be duplicate fields in the schema. `pyarrow.json.ParseOptions` should throw if there's any duplicate field in the `explicit_schema`.
I can send an MR, I'm just wondering if there's anywhere else where we do similar checks?
```python
import io
import pyarrow as pa
import pyarrow.json
SCHEMA = pa.schema(
[
pa.field("foo", pa.bool_()),
pa.field("foo", pa.bool_()),
]
)
with io.BytesIO(b'{"foo": true,"other": "bar"}') as buffer:
buffer.seek(0)
pyarrow.json.read_json(
buffer,
parse_options=pyarrow.json.ParseOptions(explicit_schema=SCHEMA),
)
```
```
pyarrow.json.read_json(
File "pyarrow/_json.pyx", line 308, in pyarrow._json.read_json
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Failed to convert JSON to bool from dictionary
```
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.