apache / apache/gluten

[VL] Incorrect result with small Parquet pages + Velox aggregation pushdown

Open
#10,366 1 comment 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

Found in Gluten + Velox when running [ClickBench](https://benchmark.clickhouse.com) Q31. This is reproduce-able with the default Gluten / Spark configurations. No specific options are required to set.

The minimal SQL to reproduce:

```sql
SELECT SearchEngineID, SUM(IsRefresh) FROM hits WHERE SearchPhrase <> '' AND (SearchEngineID in (1, 2, 4)) AND ClientIP = -807147100 GROUP BY SearchEngineID
```

Data:

`hits.parquet` from https://github.com/ClickHouse/ClickBench/?tab=readme-ov-file#data-loading

Spark answer:

```
+--------------+--------------+
|SearchEngineID|sum(IsRefresh)|
+--------------+--------------+
| 1| 1|
| 2| 35|
| 4| 3|
+--------------+--------------+
```

Gluten (Velox) answer:

```
+--------------+--------------+
|SearchEngineID|sum(IsRefresh)|
+--------------+--------------+
| 4| 1|
| 2| 37|
| 1| 1|
+--------------+--------------+
```

### Gluten version

Latest

### Spark version

3.5

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.