apache / apache/gluten

[VL] Build gluten on aarch64 with VCPKG

Open
#7,664 12 comments 0 reactions 0 assignees View on GitHub
triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Problem description

I'm trying to build gluten and velox on aarch64 architecture. I'm running the script `./dev/builddeps-veloxbe.sh --enable_vcpkg=ON --spark_version=3.5` . I get the error below. I see that these options `-mavx2 -mfma -mavx -mf16c -mlzcnt -mbmi2` are only for x86_64, so they won't work on aarch64. Do you know how I can build gluten for aarch64 please? (In version 1.1.x.x., we were able to build for both aarch64 and x86_64).

I see vcpkg `init.sh` has `VCPKG_TRIPLET=x64-linux-avx` https://github.com/apache/incubator-gluten/blob/c82af60a635fb52bf1314cd831f5915dd37d910d/dev/vcpkg/init.sh#L10. It makes sense that the `x64` and the `avx` parts of this triplet will cause errors running on aarch64. I see it got changed in https://github.com/apache/incubator-gluten/commit/f00e4536bc7ccc56a2eee5800084319130cbf806#diff-4a8a09cebf7ed92cb7321136b9ef223c4653757270e77e942fd6fe2dca30793e - do you recommend that we should cherry-pick that commit (and a few more commits that it depends on)?

```
CMake Error at /usr/local/share/cmake-3.28/Modules/CMakeTestCCompiler.cmake:67 (message):
The C compiler

"/usr/bin/cc"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: '/home/circleci/project/dev/vcpkg/.vcpkg/buildtrees/detect_compiler/x64-linux-avx-rel/CMakeFiles/CMakeScratch/TryCompile-9obttD'

Run Build Command(s): /usr/bin/ninja -v cmTC_32844
[1/2] /usr/bin/cc -fPIC -mavx2 -mfma -mavx -mf16c -mlzcnt -mbmi2 -o CMakeFiles/cmTC_32844.dir/testCCompiler.c.o -c /home/circleci/project/dev/vcpkg/.vcpkg/buildtrees/detect_compiler/x64-linux-avx-rel/CMakeFiles/CMakeScratch/TryCompile-9obttD/testCCompiler.c
FAILED: CMakeFiles/cmTC_32844.dir/testCCompiler.c.o
/usr/bin/cc -fPIC -mavx2 -mfma -mavx -mf16c -mlzcnt -mbmi2 -o CMakeFiles/cmTC_32844.dir/testCCompiler.c.o -c /home/circleci/project/dev/vcpkg/.vcpkg/buildtrees/detect_compiler/x64-linux-avx-rel/CMakeFiles/CMakeScratch/TryCompile-9obttD/testCCompiler.c
cc: error: unrecognized command line option ‘-mavx2’
cc: error: unrecognized command line option ‘-mfma’
cc: error: unrecognized command line option ‘-mavx’
cc: error: unrecognized command line option ‘-mf16c’
cc: error: unrecognized command line option ‘-mlzcnt’
cc: error: unrecognized command line option ‘-mbmi2’
ninja: build stopped: subcommand failed.

CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:11 (enable_language)
```

### System information

Velox System Info v0.0.2
Commit: 69a6bb9602c97050cc3459a09c846b84182ecb9c
CMake Version: 3.28.3
System: Linux-5.15.0-1070-aws
Arch: aarch64
CPU Name: Model name: Neoverse-N1
C++ Compiler: /usr/bin/c++
C++ Compiler Version: 9.4.0
C Compiler: /usr/bin/cc
C Compiler Version: 9.4.0
CMake Prefix Path: /usr/local;/usr;/;/usr/local;/usr/local;/usr/X11R6;/usr/pkg;/opt

\nThe results will be copied to your clipboard if xclip is installed.

### CMake log

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.