Relax pyarrow.compute.is_in type requirement
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
### Describe the enhancement requested
In the following example:
```
>>> a = pa.array([1,2,3], type=pa.int8())
>>> pa.compute.is_in(a, pa.array([255], type=pa.int64()))
Traceback (most recent call last):
File "", line 1, in
File "C:\ps_0310_is\lib\site-packages\pyarrow\compute.py", line 256, in wrapper
return func.call(args, options, memory_pool)
File "pyarrow\_compute.pyx", line 355, in pyarrow._compute.Function.call
File "pyarrow\error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow\error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Integer value 255 not in range: -128 to 127
>>>
```
It would be nice if this could return false instead of raising an error, given that its impossible for 255 to be in an int8 array. This would be more in line with isin in Pandas as well:
```
>>> a = pd.Series([1,2,3], dtype="int8")
>>> a.isin([255])
0 False
1 False
2 False
dtype: bool
>>>
```
### Component(s)
Python
Contributor guide
Research direction
Start by reproducing the pyarrow.compute.is_in example with an int8 array and an int64 value of 255, then trace the is_in entry point and its type validation. Done means out-of-range values return false instead of raising while normal membership checks continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100