apache / apache/datafusion-comet
Comet aggregation task crashes with `offset overflow`
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### Describe the bug
Was working on the experiment involving multiphase aggregation and Comet tasks failed with
```
org.apache.comet.CometNativeException: offset overflow, buffer size > 2147483647
```
The query plan
```
CometNativeColumnarToRow
+- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometFilter
+- CometExpand
+- CometProject
+- CometFilter
+- CometNativeScan parquet
Comet accelerated 17 out of 17 eligible operators (100%). Final plan contains 1 transitions between Spark and Comet.
```
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.