[VL] Poor Shuffle Read Perf in TPCH
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
### Backend
Velox
### Perf description
When switching from tpch 1T to tpch 3T, the Shuffle Read time of q21 will significantly increase, and the system's disk/network resources are not fully utilized.
In the DAG visualization, it can be seen that the time taken by the InputIterator in the 3T dataset is 60 times that of the 1T dataset.


In the Timeline, it can also be seen that the ShuffleRead time is unreasonable.

I observed the system's disk and network usage and found that the utilization is not high. I created a flame graph that includes sleep time, and it can be seen that the time taken by the LowCopyFileSegmentJniByteInputStream.read function has significantly increased.
1T:

3T:

### Spark version
Spark-3.3.2
### Spark configurations
spark.executor.cores 4
spark.executor.memory 2G
spark.executor.memoryOverhead 1g
spark.memory.offHeap.size 10g
spark.gluten.sql.columnar.backend.velox.maxSpillFileSize 1073741824
Contributor guide
Research direction
Start with the TPCH q21 comparison described in the issue, using the stated Spark 3.3.2 configuration and 1T versus 3T datasets. Profile InputIterator and LowCopyFileSegmentJniByteInputStream.read, then compare ShuffleRead time with disk and network utilization. Done means the 3T shuffle-read regression is explained and addressed or narrowed to a specific subsystem.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- backend, data, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100