TPC-DS shows bad performance on Arm server when data on HDFS
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
Hi team, I have build gluten(volex) with enable_hdfs=on on Arm server.
Generate data set on HDFS, and run TPC-DS benchmark, both steps are using tools/workload/tpcds included in Gluten repo.
But it show poor performance on Arm server when using Gluten.
I found that Gluten depends on libhdfs3, but it has been not updated for nearly 10 years.
So can I ask whether libhdfs3 downgrade the TPC-DS performance when Gluten running in Arm server and data located on HDFS?
Thanks.
Contributor guide
Assessment
This issue has not been assessed yet.