[C++] Performance of numeric casts
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
A simple numeric cast in pyarrow / Arrow C++ seems to be quite a bit slower compared to numpy. Often Arrow does more work (safe casting, handling nulls, ..), but I also see this this for a plain unsafe int64 -> float64 cast of a 1d array without nulsl, where Arrow is almost 4-5x slower than numpy. For a simple operation like that, this seems an unexpected big difference.
The python code I used to compare both:
```python
import numpy as np
import pyarrow as pa
nrows = 10**5
arr_np = np.random.randn(nrows*20).astype("int64")
arr = pa.array(arr_np)
%timeit arr_np.astype("float64")
# 1.1 ms ± 2.69 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
%timeit arr.cast(pa.float64(), safe=False)
# 4.7 ms ± 10.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
```
I see this both with my local development version as with released conda packages.
Contributor guide
Assessment
This issue has not been assessed yet.