apache / apache/datafusion-comet
Avoid `CometColumnarExchange` when next query stage requires `CometColumnarToRow`
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### What is the problem the feature request solves?
The golden file at `spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1.native_datafusion/extended.txt` has this section:
```
: : : +- HashAggregate
: : : +- CometColumnarToRow
: : : +- CometColumnarExchange
: : : +- HashAggregate
```
Presumably the first `HashAggregate` shown here is not supported by Comet (fallback reasons are missing for some reason) so this query stage needs to call `CometColumnarToRow`.
There is _probably_ no benefit in using columnar shuffle in the child stage if the parent stage immediately converts back to row-based, although we should confirm that assumption.
### Describe the potential solution
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.