[EPIC] Improved Externalized / Spilling / Large than Memory Hash Aggregation

This is a collection of items to improve external (spilling) aggregation

### Background
> Abstract—Analytical database systems offer high-performance in-memory aggregation. If there are many unique groups, temporary query intermediates may not fit RAM, requiring the use of external storage. However, switching from an in-memory to an external algorithm can degrade performance sharply

- [Robust External Hash Aggregation
in the Solid State Age (DuckDB external aggregation paper)](https://hannes.muehleisen.org/publications/icde2024-out-of-core-kuiper-boncz-muehleisen.pdf))

DataFusion has supported memory limited / spilling hash aggregation since @kazuyukitanimura added it last year in https://github.com/apache/datafusion/pull/7400.

We can likely improve this feature and @2010YOUY01 is considering working on it

### Tasks the solution you'd like

- [x] https://github.com/apache/datafusion/pull/13090
- [x] https://github.com/apache/datafusion/issues/14078
- [ ] https://github.com/apache/datafusion/issues/14851

Start by reviewing the remaining linked issue 14851 and the external aggregation paper referenced in this epic. The completed links provide context, while the open item defines the remaining work; update the checklist when that item is resolved.

Written by the indexing model from the issue text.