[C++] Allow safe cast from int64 to float64 in compute kernel for numerics that can be accurately represented above 2^53
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
### Describe the enhancement requested
See discussion found in https://github.com/apache/arrow/issues/34901.
To summarize, safe casting of int64 to float64 for numerics greater than 2^53 is currently not supported in Arrow. However, according to the Double-precision floating point format[1] defined in the IEEE 754-2008 standard, there are certain integers that can be safely cast to floating point. From 2^54 to 2^55, the representable numbers are every 2^1. From 2^54 to 2^55, the representable numbers are every 2^2. From 2^55 to 2^56, the representable numbers are every 2^3, etc.
For example, integer 18014398509481984 can be safely cast between int64 and float64 according to the IEEE 754-2008 standard, but not in arrow. (However, integer 18014398509481983 CANNOT be cast safely).
```
>>> arr = pa.array([18014398509481984], type=pa.int64())
>>> arr.cast(pa.float64())
Traceback (most recent call last):
...
pyarrow.lib.ArrowInvalid: Integer value 18014398509481984 not in range: -9007199254740992 to 9007199254740992
```
[1] https://en.wikipedia.org/wiki/Double-precision_floating-point_format
### Component(s)
C++
Contributor guide
Research direction
Start with the discussion in issue 34901 and the C++ compute kernel's current int64-to-float64 safe-cast behavior. Verify IEEE 754 representability boundaries using the examples in this issue; done means representable int64 values above 2^53 are accepted while values such as 18014398509481983 remain rejected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100