apache / apache/arrow

replace_with_mask does not properly handle chunked target array

Open
#40,780 0 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

`pyarrow.compute.replace_with_mask` requires that the inputs not be chunked arrays. The default calling pattern works as expected:
```
>>> pa.compute.replace_with_mask(pa.array([False]), pa.array([True]), pa.array([True]))

[
true
]
```
while specifying the replacement values as a chunked array raises a comprehensible exception:
```
>>> pa.compute.replace_with_mask(pa.array([False]), pa.array([True]), pa.chunked_array([[True]]))
Traceback (most recent call last):
File "/home/coder/.conda/envs/rapids/lib/python3.10/code.py", line 90, in runcode
exec(code, self.locals)
File "", line 1, in
File "/home/coder/.conda/envs/rapids/lib/python3.10/site-packages/pyarrow/compute.py", line 246, in wrapper
return func.call(args, None, memory_pool)
File "pyarrow/_compute.pyx", line 385, in pyarrow._compute.Function.call
File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Replacements must be array or scalar, not ChunkedArray([
[
true
]
])
```
However, specifying the values as a chunked array instead silently produces an invalid output:
```
>>> pa.compute.replace_with_mask(pa.chunked_array([[False]]), pa.array([True]), pa.array([True]))

[

]
```

The latter case should also be handled by the same error-checking logic used to validate the replacements (unless chunked arrays can in fact be supported, which would be nice but isn't too important).

I observe this with pyarrow 15.0.2 installed via conda (from conda-forge) on Ubuntu 22.04.
### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.