apache / apache/gluten

[CH] Adjust celeborn shuffle write time

Open
#4,338 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

CH (ClickHouse)

### Bug description

For the current implementation of ch celeborn shuffle write, native's spill time includes native's write time which leads to inaccurate metrics.
There may be something like below:
![image](https://github.com/oap-project/gluten/assets/19429353/3f486e59-fe5b-4843-85fa-583685effae9)
I think native's write time is enough now.

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.