apache / apache/datafusion

Enable `enable_row_number_to_aggregate` by default

Open
#24,761 1 comment 0 reactions 1 assignee Claimed by @saadtajwar View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

In PR #23824, `ReplaceFilterTop1` rewrites `Filter(row_number() = 1)` over a `PARTITION BY` window into `Aggregate(first_value(...) GROUP BY partition)`. The rewrite is currently behind the config `optimizer.enable_row_number_to_aggregate`, which currently defaults to false - this was originally due to #23601 not being merged in yet, but as #23601 is now closed out it would be helpful for this to be enabled by default

### Describe the solution you'd like

To flip `enable_row_number_to_aggregate` to true:

- Confirm nested/wide types no longer hit the per-group Accumulator path, and measure the remaining GenericValueState cost vs primitives.
- Benchmark Window+Filter vs the rewritten Aggregate across partition cardinality, payload width, and value types (primitive, string, nested)
- Confirm there are no remaining correctness gaps for the rewrite
- If results look good, set the default to true, drop the "disabled by default" note in `config.rs`, and regenerate `docs/source/user-guide/configs.md` plus `information_schema.slt`.

### Describe alternatives you've considered

Keep off by default (wouldn't be very useful for optimization)

### Additional context

See original issue: #23603 & original PR: #23824 & associated epic: #23600

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.