facebookincubator / facebookincubator/velox

Spark aggregation fuzzer crashes streaming aggregation

Open
#16,632 7 comments 1 reaction 0 assignees View on GitHub
fuzzer spark-functions
Dominant language
C++
Stars
4.2k
Forks
1.6k
Avg merge
3d 3h
Merged PRs (30d)
246

Description

## Summary

The Spark aggregation fuzzer consistently crashes (SIGSEGV at address 0x0) when testing the `first` aggregate function with complex ROW-typed group-by keys in a specific plan variant: **FINAL STREAMING aggregation over LocalMerge with forced abandon-partial-aggregation** (plan #14 in the fuzzer).

## Reproduction

The crash has been confirmed across multiple CI runs with different seeds:

| CI Run | Initial Seed | Crash Iteration | Crash Seed | Input Type | Group-by Keys |
|--------|-------------|-----------------|------------|------------|---------------|
| [22654073141](https://github.com/facebookincubator/velox/actions/runs/22654073141) | 5500 | 309 | 3776397994 | BIGINT | ROW, ROW, ROW, BIGINT |
| [22636891751](https://github.com/facebookincubator/velox/actions/runs/22636891751) | 9886 | 170 | 1997082133 | REAL | ROW(7 fields), ROW(6 fields), ARRAY |

## Failing Plan Shape

The crash always occurs on the same plan structure (plan #14 in the fuzzer):

```
Aggregation[FINAL STREAMING [g0, g1, ...] a0 := first("a0")]
LocalMerge[g0 ASC, g1 ASC, ...]
Aggregation[PARTIAL STREAMING [...] a0 := first(ROW["c0"])]
OrderBy[g0 ASC, g1 ASC, ...]
Project[...]
Values[300 rows in 3 vectors]
Aggregation[PARTIAL STREAMING [...] a0 := first(ROW["c0"])]
OrderBy[...]
Project[...]
Values[300 rows in 3 vectors]
... (4 parallel sources)
```

The crash specifically occurs during the **"forced abandon-partial-aggregation"** test variant of this plan.

## Crash Details

- Signal: SIGSEGV (Signal 11) at address 0x0 (NULL pointer dereference)
- The crash is in the main thread
- Exit code: 139
- No stack trace available in CI binary (lacks debug symbols)

## Analysis

- The `first` aggregate function itself appears to work correctly for simpler plan variants (single aggregation, partial→final, partial→intermediate→final, and even streaming with spilling).
- The crash is specific to the combination of: streaming aggregation + LocalMerge + abandon-partial-aggregation.
- The bug is likely in the aggregation framework's handling of the abandon-partial path for streaming aggregation with LocalMerge, not in the `first` function implementation itself.
- Complex ROW-typed group-by keys may be a contributing factor (possibly related to comparison or serialization of complex types during the merge).

## Workaround

Adding `first` to the `skipFunctions` set in `SparkAggregationFuzzerTest.cpp` will prevent the crash until the root cause is fixed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.