bytedance / bytedance/bolt

[Bug] Spark float-to-integer casts return minimum at positive overflow boundary

Closed Beginner friendly
#809 0 comments 0 reactions 0 assignees View on GitHub
bug needs triage
Dominant language
C++
Stars
177
Forks
107
Avg merge
3d 14h
Merged PRs (30d)
55

Description

### Component Selection

- [x] Core Engine (Expression eval, Memory, Vector)
- [ ] Connectors / File Formats (Hive, Parquet, etc.)
- [ ] API / Bindings (Python, etc.)
- [ ] Build
- [ ] Other

### Describe the Bug

In Spark-compatible mode, Bolt returns incorrect results when a
runtime-computed floating-point value is exactly the first value outside the
positive range of the destination integer type:

- `FLOAT 2^31` cast to `INT`
- `DOUBLE 2^63` cast to `BIGINT`

Java Spark saturates these values to the destination type's maximum value.
Bolt native instead returns the destination type's minimum value.

The expressions intentionally depend on `id` so Catalyst cannot constant-fold
the casts before native execution.

### Reproduction Steps

Run with ANSI mode disabled:

```sql
SET spark.sql.ansi.enabled=false;

WITH src AS (
SELECT id FROM range(0, 2)
)
SELECT
id,
CAST(
CAST(
CAST(id AS FLOAT) * CAST('2147483648' AS FLOAT)
AS FLOAT
)
AS INT
) AS float_to_int,
CAST(
CAST(id AS DOUBLE) * CAST('9223372036854775808' AS DOUBLE)
AS BIGINT
) AS double_to_bigint
FROM src
ORDER BY id;
```

Bolt native result:

```text
id float_to_int double_to_bigint
0 0 0
1 -2147483648 -9223372036854775808
```

Java Spark result:

```text
id float_to_int double_to_bigint
0 0 0
1 2147483647 9223372036854775807
```

### Bolt Version / Commit ID

`8040d32a87d951349544089134a66cf91e86e214`

### System Configuration

- **OS**: Linux
- **Compiler**: Clang 22 with libc++
- **Build Type**: Release
- **CPU Arch**: x86_64
- **Framework**: Spark, comparing Bolt native execution with Java Spark execution

### Expected Behavior

Bolt's Spark-compatible cast should match Java Spark:

```text
id float_to_int double_to_bigint
0 0 0
1 2147483647 9223372036854775807
```

### Additional context

The relevant path appears to be the floating-point-to-integral conversion in
`bolt/expression/CastExpr-tpl.cpp`.

For these destination types, converting
`std::numeric_limits::max()` to the source floating-point type rounds
it up:

- `INT_MAX` becomes `FLOAT 2^31`
- `INT64_MAX` becomes `DOUBLE 2^63`

As a result, a strict comparison such as
`from > std::numeric_limits::max()` does not identify the exact `2^N`
boundary as overflow after the usual conversion. The subsequent out-of-range
C++ floating-point-to-integer conversion is undefined behavior; on this x86_64
build it produces the integer minimum value.

Contributor guide

Open the contributing guide

Research direction

Start in bolt/expression/CastExpr-tpl.cpp and trace the floating-point-to-integral conversion used by Spark-compatible casts. Run the SQL reproduction with ANSI mode disabled, then verify that the positive overflow boundaries for INT and BIGINT produce the destination maximum values rather than minimum values.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.