apache / apache/gluten

[VL] Support partial generate for UDTF

Open
#10,472 1 comment 0 reactions 1 assignee Claimed by @jiangjiangtian View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

For generator that is an instance of `HiveGenericUDTF`, the `GenerateExec` can't be offloaded and therefore the plan will have a c2r and r2c. After the evaluation of `GenerateExec`, the number of rows may be several times more than that before the evaluation of `GenerateExec`. So we can implement `partial generate` to improve performance.

### Gluten version

None

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.