apache / apache/datafusion

CI: extended tests don't run on changes to functions-aggregate / functions-window, which the aggregate and window fuzzers exercise

Open Beginner friendly
#24,212 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

## Is your feature request related to a problem or challenge?

The fuzz tests in `datafusion/core/tests/fuzz_cases/` are gated behind the `extended_tests` feature, which only `.github/workflows/extended.yml` enables. On pull requests that workflow is limited to these paths:

```yaml
paths:
- 'datafusion/physical*/**/*.rs'
- 'datafusion/expr*/**/*.rs'
- 'datafusion/optimizer/**/*.rs'
- 'datafusion/sql/**/*.rs'
- 'datafusion-testing'
```

Two of the fuzzers exercise crates that aren't on that list:

- `aggregate_fuzz.rs` builds queries with `.with_aggregate_function(...)` for **`min`, `max`, `sum`, `count`, `median`, `first_value`, `last_value`** — implemented in `datafusion/functions-aggregate/`.
- `window_fuzz.rs` imports `datafusion_functions_window` and `datafusion_functions_aggregate`.

So a PR that changes an aggregate or window function implementation does not run the fuzz suite built to exercise it. It is still caught by the post-merge run on `main`, so this is a *gating* gap rather than missing coverage — the cost is a red `main` and a revert instead of a red PR.

**A recent concrete case:** #24104 *"fix: support untyped NULL input for median"* (merged 2026-08-09) changed `datafusion/functions-aggregate/src/median.rs`, and was itself fixing a regression between DataFusion 53 and 54. The workflows that ran on its head SHA were CodeQL, Dependencies, Detect breaking changes, Dev, Labeler, Large files PR check, and Rust; `Datafusion extended tests` did not run.

To be clear about what that does and doesn't show: `aggregate_fuzz` generates typed columns with a null percentage, whereas that bug involved an untyped `NULL` literal, so the fuzzer very likely would **not** have caught this particular issue. The point is narrower — a fuzzer-covered function was modified and its fuzzer didn't run.

## Describe the solution you'd like

Add the two crates the fuzzers actually import:

```yaml
- 'datafusion/functions-aggregate/**/*.rs'
- 'datafusion/functions-window/**/*.rs'
```

**Measured cost.** Over the last 50 merged PRs, **2 (4%)** would newly trigger the workflow. On a recent `main` run (`31366139826`) the three jobs run in parallel: sqlite suite 9m32s, `extended_tests` 9m21s, hash collisions 5m20s — about 9.5 minutes wall-clock, ~24 runner-minutes.

I deliberately kept `datasource*` and `catalog*` out of this proposal even though some fuzz modules import them: 9 of the 12 unfiltered PRs in that sample were `datasource*`, and six were a single one-off `refactor(proto): migrate Source serde` batch, so including them would raise the trigger rate to ~24% for what looks like a temporary campaign.

## Describe alternatives you've considered

1. **Per-job path conditions** — put the wider path set on `linux-test-extended` only, leaving the sqlite and hash-collision jobs on the current narrow trigger. Same wall-clock, but ~9 runner-minutes instead of ~24. More surgical, slightly more complex than a two-line paths addition.
2. **Do nothing.** Post-merge detection on `main` may already be the intended trade-off, given that #19228's motivation for `extended_tests` was local developer inner-loop speed rather than CI cost. If that's the case it would be useful to have it stated, since the path list reads as a deliberate "most likely to trigger errors" selection.

## Additional context

Verified locally at `308e212`: `cargo test -p datafusion --features extended_tests --profile ci --test fuzz --no-run` takes 2m09s cold, and all 114 fuzz tests pass in 4m48s (`--test-threads=4`). `aggregate_fuzz` is 10 tests, `window_fuzz` is 2; together they run in 39.6s. So the two fuzzers relevant here are a small fraction of the suite.

I'm happy to open a PR for whichever option you prefer.

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/extended.yml and inspect the existing paths filter for the extended-tests workflow. Add coverage for the two functions crate paths, then verify the workflow configuration and confirm changes under those paths trigger the extended tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, rust
Domain
ci-cd
Issue type
Feature
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.