apache / apache/arrow

[Python] pa.array([Decimal("nan")]) raises

Open
#35,717 6 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

```
pa.array([Decimal("nan")])

Traceback (most recent call last):
File "", line 1, in
File "pyarrow/array.pxi", line 320, in pyarrow.lib.array
File "pyarrow/array.pxi", line 39, in pyarrow.lib._sequence_to_array
File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Decimal precision out of range [1, 38]: -2147483648
```

Based on how pyarrow handles numpy nans I'd expect this to get converted to a pyarrow null.

pyarrow 11.0.0 on mac

### Component(s)

Python

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with pyarrow 11.0.0 and Decimal("nan"), then compare it with NumPy NaN handling. Start from the conversion path named in pyarrow/array.pxi and the error reported through pyarrow/error.pxi; done means Decimal NaN becomes a PyArrow null and the behavior is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.