apache / apache/gluten

[BUG] max(BIGINT) Returns Wrong Value with High Cardinality Input and Velox Engine

Open
#10,292 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Backend

VL (Velox)

### Bug description

When running a GROUP BY query with max(ts) on a BIGINT column, Gluten 1.3 with the Velox backend returns an incorrect result for some inputs.
example:
```
select
uid,
max(msg_ts) as last_msg_ts
from
test_parquet_max
where uid = '710726744'
group by uid
```
excepted result:
uid: 710726744 last_msg_ts: 1749185469637
Actual result
uid: 710726744 last_msg_ts: 140485873811016

veloxplan:
```
I20250730 10:10:50.677613 28032 VeloxRuntime.cc:145] ############### Velox plan for task [Stage: 0 TID: 5] ###############
-- Project[3][expressions: (n3_2:INTEGER, hash_with_seed(42,"n1_5")), (n3_3:BIGINT, "n1_5"), (n3_4:BIGINT, "n2_1")] -> n3_2:INTEGER, n3_3:BIGINT, n3_4:BIGINT
-- Aggregation[2][PARTIAL [n1_5] n2_1 := max_partial("n1_4")] -> n1_5:BIGINT, n2_1:BIGINT
-- Project[1][expressions: (n1_4:BIGINT, "n0_0"), (n1_5:BIGINT, "n0_1")] -> n1_4:BIGINT, n1_5:BIGINT
-- TableScan[0][table: hive_table, remaining filter: (and(isnotnull("uid"),equalto("uid",710726744)))] -> n0_0:BIGINT, n0_1:BIGINT, n0_2:VARCHAR, n0_3:VARCHAR

I20250730 10:10:54.165143 28032 VeloxRuntime.cc:145] ############### Velox plan for task [Stage: 2 TID: 20] ###############
-- Aggregation[1][SINGLE [n0_0] n1_1 := max_merge_extract("n0_1")] -> n0_0:BIGINT, n1_1:BIGINT
-- ValueStream[0][] -> n0_0:BIGINT, n0_1:BIGINT
```

### Gluten version

Gluten-1.3

### Spark version

Spark-3.5.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the GROUP BY query with the Velox backend and inspect the Velox plan and logs emitted by VeloxRuntime.cc. Compare the partial max and max_merge_extract stages against the expected BIGINT result. Done means the query returns the correct maximum value for the reported high-cardinality input.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.