[C++][Python] pa.array raises for mixed scalar types (float16 + int)
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
pa.array raises with mixed scalars for np.float16 and a numpy int:
```java
In [1]: import numpy as np
...: import pyarrow as pa
...: pa.array([np.float16(1.5), np.int64(5)])
---------------------------------------------------------------------------
ArrowInvalid Traceback (most recent call last)
in
1 import numpy as np
2 import pyarrow as pa
----> 3 pa.array([np.float16(1.5), np.int64(5)])
~/git_repo/arrow/python/pyarrow/array.pxi in pyarrow.lib.array()
269 else:
270 # ConvertPySequence does strict conversion if type is explicitly passed
--> 271 return _sequence_to_array(obj, mask, size, type, pool, c_from_pandas)
272
273
~/git_repo/arrow/python/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()
38
39 with nogil:
---> 40 check_status(ConvertPySequence(sequence, mask, options, &out))
41
42 if out.get().num_chunks() == 1:
~/git_repo/arrow/python/pyarrow/error.pxi in pyarrow.lib.check_status()
82
83 if status.IsInvalid():
---> 84 raise ArrowInvalid(message)
85 elif status.IsIOError():
86 # Note: OSError constructor is
ArrowInvalid: Cannot mix NumPy dtypes float16 and int64
```
On master I get the same behaviour with all numpy ints (signed and unsigned)
**Reporter**: [Andrew Wieteska](https://issues.apache.org/jira/browse/ARROW-9817) / @arw2019
**Note**: *This issue was originally created as [ARROW-9817](https://issues.apache.org/jira/browse/ARROW-9817). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Reproduce the failure with the NumPy float16 and int64 values shown in the issue, then start at python/pyarrow/array.pxi, especially pyarrow.lib.array(), _sequence_to_array(), and ConvertPySequence. Done means pa.array accepts the mixed scalar sequence without raising ArrowInvalid while preserving the expected array values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100