abetlen / abetlen/llama-cpp-python

Can't install llama-cpp-python with HIPBLAS/ROCm on Windows

Đang mở
#1,489 8 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

I have a RX 6900XT GPU, and after installing ROCm 5.7 I followed the instructions to install llama-cpp-python with HIPBLAS=on, but got the error of "Building wheel for llama-cpp-python (pyproject.toml) did not run successfully".

Full error log: [llama-cpp-python-hipblas-error.txt](https://github.com/abetlen/llama-cpp-python/files/15449465/llama-cpp-python-hipblas-error.txt)

As with the previously closed but unaddressed #1009, my debugging efforts have led me to believe that the wrong C and C++ compilers are being chosen for the cmake build:

1. MSVC is selected instead of clang

```cmake
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.39.33523.0
-- The CXX compiler identification is MSVC 19.39.33523.0
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - works
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - works
-- Detecting CXX compile features
-- Detecting CXX compile features - done
```

2. A [clang-only option](https://clang.llvm.org/docs/ClangCommandLineReference.html) ('-x') is then ignored during compilation
```cmake
ClCompile:
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64\CL.exe /c /I"C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\." /nologo /W1 /WX- /diagnostics:column /O2 /Ob2 /D _MBCS /D WIN32 /D _WINDOWS /D NDEBUG /D GGML_SCHED_MAX_COPIES=4 /D GGML_USE_LLAMAFILE /D GGML_USE_HIPBLAS /D GGML_USE_CUDA /D GGML_CUDA_DMMV_X=32 /D GGML_CUDA_MMV_Y=1 /D K_QUANTS_PER_ITERATION=2 /D _CRT_SECURE_NO_WARNINGS /D _XOPEN_SOURCE=600 /D __HIP_PLATFORM_HCC__=1 /D __HIP_PLATFORM_AMD__=1 /D "CMAKE_INTDIR=\"Release\"" /Gm- /EHsc /MD /GS /arch:AVX2 /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /std:c11 /Fo"ggml.dir\Release\\" /Fd"ggml.dir\Release\ggml.pdb" /external:W0 /Gd /TC /errorReport:queue /external:I "C:/Program Files/AMD/ROCm/5.7/include" -x hip "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml.c" "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml-alloc.c" "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml-backend.c" "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml-quants.c"
cl : command line warning D9002: ignoring unknown option '-x' [C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj]
```

3. The subsequent argument ('hip') is interpreted as a non-existent source file
```cmake
/hip(1,1): error C1083: Cannot open source file: 'hip': No such file or directory [C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj]
(compiling source file '/hip')
```
4. The build fails
```cmake
"C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\ALL_BUILD.vcxproj" (default target) (1) ->
"C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj" (default target) (4) ->
(ClCompile target) ->
/hip(1,1): error C1083: Cannot open source file: 'hip': No such file or directory [C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj]

1 Warning(s)
1 Error(s)

Time Elapsed 00:00:02.54


*** CMake build failed
```

As with the original reporter, I've also tried setting CMake environment variables to force Clang compilation, with no change in result:

`[Environment]::SetEnvironmentVariable('CMAKE_ARGS', "-DLLAMA_HIPBLAS=on -DCMAKE_CXX_COMPILER='C:/Program Files/AMD/ROCm/5.7/bin/clang++.exe' -DCMAKE_C_ABI_COMPILED=FALSE -DCMAKE_CXX_ABI_COMPILED=FALSE -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_EXTENSIONS=OFF")`

`[Environment]::SetEnvironmentVariable('CMAKE_ARGS', "-DLLAMA_HIPBLAS=on -DCXX='C:/Program Files/AMD/ROCm/5.7/bin/clang++.exe' -DCMAKE_C_ABI_COMPILED=FALSE -DCMAKE_CXX_ABI_COMPILED=FALSE -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_EXTENSIONS=OFF")`

PS: Reading through #40 it's rather concerning that MSVC might be needed for Windows compilation. Is this still the case?

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.