nulls in input to pc.case_when cause incorrect results when `len(cond) >~ 70`
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
Issue: `pc.case_when` gives incorrect results when the input `cond` contains null struct values, and the length of `cond` is greater than ~70.
Example:
When the length of `cond` is small (in this case 2), I get what I would expect:
```python
from pyarrow import compute as pc
print(pc.case_when([{'0': True, '1': False}, {'0': None, '1': None}], [0,0], [1,1], [-1,-1]).to_pylist())
# prints [0, -1]
```
But if cond gets longer than about ~70, it starts replacing all values after a certain point with the default value. Here's the same example as above, but 50x longer (100 elements),
```python
from pyarrow import compute as pc
pc.case_when([{'0': True, '1': False}, {'0': None, '1': None}] * 50, [0,0] * 50, [1,1] *50, [-1,-1] * 50).to_pylist()
```
Expected behavior:
Should print
[0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1]
Actual behavior:
Prints (note all the -1s at the end).
[0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
Pyarrow version: 10.0.1
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.