[VL] Gluten write more shuffle data than vanilla Spark
Open
bug
triage
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
I can see that for the same record numbers, gluten always write much larger shuffle data than vanilla, why? From my understanding, gluten should use less storage since it's written in columnar format.
gluten

vanilla

### Spark version
Spark-3.2.x
### Spark configurations
_No response_
### System information
gluten version: 1.3.0
### Relevant logs
```bash
```
Contributor guide
Assessment
This issue has not been assessed yet.