[VL] Track Jvm based libhdfs performance gap issue
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
Currently, we supported JVM-based libhdfs in both the [Velox ](https://github.com/facebookincubator/velox/pull/9835)and [Gluten ](https://github.com/apache/incubator-gluten/pull/6172)communities. However, the performance in Jenkins tests has not met our expectations.
- Static linking with libhdfs3.so: 340%
- Dynamic linking with JVM: 214.75% (requires further investigation)
- Dynamic linking with libhdfs3: 323.85% (only replacing libhdfs.so with libhdfs3.so)
- Expected performance for static linking in [PR#7697](https://github.com/apache/incubator-gluten/pull/7697): 340%
Contributor guide
Assessment
This issue has not been assessed yet.