apache / apache/gluten

TPC-DS shows bad performance on Arm server when data on HDFS

Open
#5,629 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

Hi team, I have build gluten(volex) with enable_hdfs=on on Arm server.
Generate data set on HDFS, and run TPC-DS benchmark, both steps are using tools/workload/tpcds included in Gluten repo.
But it show poor performance on Arm server when using Gluten.
I found that Gluten depends on libhdfs3, but it has been not updated for nearly 10 years.
So can I ask whether libhdfs3 downgrade the TPC-DS performance when Gluten running in Arm server and data located on HDFS?

Thanks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.