[VL] Result mismatch in cast(float|double as string)
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
Velox always produce fixed precision scientific notation when cast number less equal `1E-3` or greater equal `1E7`. But vanilla spark produce shortest scientific notation.
```
cast(double as string)
5.957E-4 -> 5.9570000000000001E-4
1.0E-6 -> 9.9999999999999995E-7
7.639E-4 -> 7.6389999999999997E-4
```
https://github.com/facebookincubator/velox/pull/12574 may fix fixed-precision issue, but the results are still mismatch due to https://bugs.openjdk.org/browse/JDK-4511638. Double.toString in java < 19 may product incorrect and longer result, for example:
| Double | Double.toString JDK <= 18 | JDK 19, Other IEEE 754 C++ Implements | |
| -- | -- | -- | -- |
| 7.5371334E25 | 7.5371335E25 | 7.5371334E25 | # incorrect |
| _carrier uint64_ | _4994259165719689788_ | _4994259165603274466_ | |
| 1.0E23 | 9.999999999999999E22 | 1.0E23 | # longer |
| _carrier uint64_ | _4950912855330343670_ | _4950912855330343670_ | |
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Assessment
This issue has not been assessed yet.