apache / apache/gluten

[VL] Result mismatch in cast(float|double as string)

Open
#8,959 2 comments 1 reaction 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

Velox always produce fixed precision scientific notation when cast number less equal `1E-3` or greater equal `1E7`. But vanilla spark produce shortest scientific notation.

```
cast(double as string)
5.957E-4 -> 5.9570000000000001E-4
1.0E-6 -> 9.9999999999999995E-7
7.639E-4 -> 7.6389999999999997E-4
```

https://github.com/facebookincubator/velox/pull/12574 may fix fixed-precision issue, but the results are still mismatch due to https://bugs.openjdk.org/browse/JDK-4511638. Double.toString in java < 19 may product incorrect and longer result, for example:

| Double | Double.toString JDK <= 18 | JDK 19, Other IEEE 754 C++ Implements | |
| -- | -- | -- | -- |
| 7.5371334E25 | 7.5371335E25 | 7.5371334E25 | # incorrect |
| _carrier uint64_ | _4994259165719689788_ | _4994259165603274466_ | |
| 1.0E23 | 9.999999999999999E22 | 1.0E23 | # longer |
| _carrier uint64_ | _4950912855330343670_ | _4950912855330343670_ | |

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.