[Bug] Spark float-to-integer casts return minimum at positive overflow boundary
- Dominant language
- C++
- Stars
- 177
- Forks
- 107
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 55
Description
### Component Selection
- [x] Core Engine (Expression eval, Memory, Vector)
- [ ] Connectors / File Formats (Hive, Parquet, etc.)
- [ ] API / Bindings (Python, etc.)
- [ ] Build
- [ ] Other
### Describe the Bug
In Spark-compatible mode, Bolt returns incorrect results when a
runtime-computed floating-point value is exactly the first value outside the
positive range of the destination integer type:
- `FLOAT 2^31` cast to `INT`
- `DOUBLE 2^63` cast to `BIGINT`
Java Spark saturates these values to the destination type's maximum value.
Bolt native instead returns the destination type's minimum value.
The expressions intentionally depend on `id` so Catalyst cannot constant-fold
the casts before native execution.
### Reproduction Steps
Run with ANSI mode disabled:
```sql
SET spark.sql.ansi.enabled=false;
WITH src AS (
SELECT id FROM range(0, 2)
)
SELECT
id,
CAST(
CAST(
CAST(id AS FLOAT) * CAST('2147483648' AS FLOAT)
AS FLOAT
)
AS INT
) AS float_to_int,
CAST(
CAST(id AS DOUBLE) * CAST('9223372036854775808' AS DOUBLE)
AS BIGINT
) AS double_to_bigint
FROM src
ORDER BY id;
```
Bolt native result:
```text
id float_to_int double_to_bigint
0 0 0
1 -2147483648 -9223372036854775808
```
Java Spark result:
```text
id float_to_int double_to_bigint
0 0 0
1 2147483647 9223372036854775807
```
### Bolt Version / Commit ID
`8040d32a87d951349544089134a66cf91e86e214`
### System Configuration
- **OS**: Linux
- **Compiler**: Clang 22 with libc++
- **Build Type**: Release
- **CPU Arch**: x86_64
- **Framework**: Spark, comparing Bolt native execution with Java Spark execution
### Expected Behavior
Bolt's Spark-compatible cast should match Java Spark:
```text
id float_to_int double_to_bigint
0 0 0
1 2147483647 9223372036854775807
```
### Additional context
The relevant path appears to be the floating-point-to-integral conversion in
`bolt/expression/CastExpr-tpl.cpp`.
For these destination types, converting
`std::numeric_limits::max()` to the source floating-point type rounds
it up:
- `INT_MAX` becomes `FLOAT 2^31`
- `INT64_MAX` becomes `DOUBLE 2^63`
As a result, a strict comparison such as
`from > std::numeric_limits::max()` does not identify the exact `2^N`
boundary as overflow after the usual conversion. The subsequent out-of-range
C++ floating-point-to-integer conversion is undefined behavior; on this x86_64
build it produces the integer minimum value.
Contributor guide
Research direction
Start in bolt/expression/CastExpr-tpl.cpp and trace the floating-point-to-integral conversion used by Spark-compatible casts. Run the SQL reproduction with ANSI mode disabled, then verify that the positive overflow boundaries for INT and BIGINT produce the destination maximum values rather than minimum values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, sql
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100