apache / apache/datafusion-comet
Broaden storage-partitioned-join tests, then enable reportPartitioning by default
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### What is the problem the feature request solves?
Follow up for #5323
The grouping report (reportPartitioning.enabled) ships off by default until we've covered more of Spark's storage-partitioned-join paths. Add tests for:
- a heavily skewed key, so the replicate path actually runs (not just empty-partition padding);
- different bucket counts per side (e.g. bucket(8) vs bucket(4)), so Spark inserts a partition reducer and the Comet scan must match the reduced grouping;
- three-way and self joins on the same key.
Once these tests pass, then we can flip `reportPartitioning.enabled` on by default
### Describe the potential solution
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.