EHN: Add a `is_castable` function and/or `errors=coerce` option to `cast`
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
I'd like to cast a `string` array to `float`, but it can contain entries that do not represent floats.
```python
import pyarrow as pa
import pyarrow.compute as pc
arr = pa.array(["1.2", "3", "10-20", None, "nan", ""])
out = pc.cast(arr, pa.float64(), safe=False) # raises ArrowInvalid
print(out) # E: [1.2, 3, null, null, nan, null]
```
My current workaround is to export to `pandas` and use [`pandas.to_numeric(errors="coerce")`](https://pandas.pydata.org/docs/reference/api/pandas.to_numeric.html#pandas-to-numeric).
However, it would be nice if `pyarrow` had some built-in machinery to deal with this situation:
1. Add a function that yields a boolean mask of all values that are castable.
```python
def is_castable(arr, target_type, options=None) -> Array[bool]:
"""Returns boolean mask of values that can be cast to target_type,
under the chosen options."""
```
Such a function would also be useful for extracting the set of all values that cannot be cast.
2. Either a `force_cast` function or a `errors={"raise", "coerce"}` option like `pandas.to_numeric` that catches conversion errors, essentially as a shortcut for
```python
def force_cast(array, options):
# 1. check which values are castable with the given options
mask = pa.is_castable(array, options)
# 2. mask out all non-castable values
array = pc.compute.where(mask, array, None)
# 3. cast the result (guaranteed to succeed)
return pc.cast(array, options)
```
### Alternatives
Of course one can emulate `is_castable` in this particular case by using a regex, but this is problematic since it needs to be synced with `pyarrow`s internal logic of when it considers a string castable to float.
### Component(s)
C++, Python
Contributor guide
Assessment
This issue has not been assessed yet.