apache / apache/gluten

[VL] inputIteratorTransformer exhibits unexpectedly high time cost

Open
#8,063 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

Currently, while executing vectorized etls, we unexpectedly found that inputIteratorTransformer is consuming a significant amount of CPU time.

The Spark UI is as follows:

image
image
image

The image shows the process from Table Scan to inputIteratorTransformer.
inputIteratorTransformer includes the total execution time of all previous operators, but from the CPU time perspective, it doesn’t seem to be the case.

Spark version: spark-3.0
Gluten version: branch-1.2

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.