apache / apache/datafusion

In general the multi group by option seems to make certain scenarios worse.

Aperta
#17,850 1 commento 0 reazioni 1 assegnatario Rivendicata da @nathanb9 Vedi su GitHub
enhancement performance
Lingua principale
Rust
Stelle
9.3k
Fork
2.4k
Merge medio
3g 11h
PR unite (30g)
360

Descrizione

In general the multi group by option seems to make certain scenarios worse.
I added a flag for easy control of whether it's used in this commit: https://github.com/ashdnazg/datafusion/commit/6a0b13bc2a5f9305b38f9bd0fa87cbf865e9c37b (see [enable-gby.patch](https://github.com/user-attachments/files/22625294/enable-gby.patch) for patch)
and then checked:
```
create or replace table foo as select (random() * 4)::integer as int_val, (random() * 4)::integer as int_val2, (random() * 4)::integer as int_val3 from generate_series(1, 1000000000);
set datafusion.execution.enable_multi_group_by to false;
select int_val, int_val2, int_val3 from foo GROUP BY int_val, int_val2, int_val3;
set datafusion.execution.enable_multi_group_by to true;
select int_val, int_val2, int_val3 from foo GROUP BY int_val, int_val2, int_val3;
```

I get `Elapsed 1.037 seconds.` with the flag set to false and `Elapsed 1.382 seconds.` with the flag set to true.

Perhaps the logic for when to use it should be more careful than just "whenever it's supported". But I suspect this PR is not the right spot for that discussion.

_Originally posted by @ashdnazg in https://github.com/apache/datafusion/issues/17726#issuecomment-3347258929_

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.