In general the multi group by option seems to make certain scenarios worse.
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
In general the multi group by option seems to make certain scenarios worse.
I added a flag for easy control of whether it's used in this commit: https://github.com/ashdnazg/datafusion/commit/6a0b13bc2a5f9305b38f9bd0fa87cbf865e9c37b (see [enable-gby.patch](https://github.com/user-attachments/files/22625294/enable-gby.patch) for patch)
and then checked:
```
create or replace table foo as select (random() * 4)::integer as int_val, (random() * 4)::integer as int_val2, (random() * 4)::integer as int_val3 from generate_series(1, 1000000000);
set datafusion.execution.enable_multi_group_by to false;
select int_val, int_val2, int_val3 from foo GROUP BY int_val, int_val2, int_val3;
set datafusion.execution.enable_multi_group_by to true;
select int_val, int_val2, int_val3 from foo GROUP BY int_val, int_val2, int_val3;
```
I get `Elapsed 1.037 seconds.` with the flag set to false and `Elapsed 1.382 seconds.` with the flag set to true.
Perhaps the logic for when to use it should be more careful than just "whenever it's supported". But I suspect this PR is not the right spot for that discussion.
_Originally posted by @ashdnazg in https://github.com/apache/datafusion/issues/17726#issuecomment-3347258929_
Contributor guide
Assessment
This issue has not been assessed yet.