abetlen / abetlen/llama-cpp-python

Can't install llama-cpp-python with HIPBLAS/ROCm on Windows

Ouverte
#1,489 8 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
10.6k
Forks
1.4k
Métriques de merge des PR
Métriques de PR en attente

Description

I have a RX 6900XT GPU, and after installing ROCm 5.7 I followed the instructions to install llama-cpp-python with HIPBLAS=on, but got the error of "Building wheel for llama-cpp-python (pyproject.toml) did not run successfully".

Full error log: [llama-cpp-python-hipblas-error.txt](https://github.com/abetlen/llama-cpp-python/files/15449465/llama-cpp-python-hipblas-error.txt)

As with the previously closed but unaddressed #1009, my debugging efforts have led me to believe that the wrong C and C++ compilers are being chosen for the cmake build:

1. MSVC is selected instead of clang

```cmake
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.39.33523.0
-- The CXX compiler identification is MSVC 19.39.33523.0
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - works
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - works
-- Detecting CXX compile features
-- Detecting CXX compile features - done
```

2. A [clang-only option](https://clang.llvm.org/docs/ClangCommandLineReference.html) ('-x') is then ignored during compilation
```cmake
ClCompile:
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64\CL.exe /c /I"C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\." /nologo /W1 /WX- /diagnostics:column /O2 /Ob2 /D _MBCS /D WIN32 /D _WINDOWS /D NDEBUG /D GGML_SCHED_MAX_COPIES=4 /D GGML_USE_LLAMAFILE /D GGML_USE_HIPBLAS /D GGML_USE_CUDA /D GGML_CUDA_DMMV_X=32 /D GGML_CUDA_MMV_Y=1 /D K_QUANTS_PER_ITERATION=2 /D _CRT_SECURE_NO_WARNINGS /D _XOPEN_SOURCE=600 /D __HIP_PLATFORM_HCC__=1 /D __HIP_PLATFORM_AMD__=1 /D "CMAKE_INTDIR=\"Release\"" /Gm- /EHsc /MD /GS /arch:AVX2 /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /std:c11 /Fo"ggml.dir\Release\\" /Fd"ggml.dir\Release\ggml.pdb" /external:W0 /Gd /TC /errorReport:queue /external:I "C:/Program Files/AMD/ROCm/5.7/include" -x hip "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml.c" "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml-alloc.c" "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml-backend.c" "C:\Users\mbamg\AppData\Local\Temp\pip-install-ps72vnaz\llama-cpp-python_1bb2b9676f39468bba7efbe70e3a1f33\vendor\llama.cpp\ggml-quants.c"
cl : command line warning D9002: ignoring unknown option '-x' [C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj]
```

3. The subsequent argument ('hip') is interpreted as a non-existent source file
```cmake
/hip(1,1): error C1083: Cannot open source file: 'hip': No such file or directory [C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj]
(compiling source file '/hip')
```
4. The build fails
```cmake
"C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\ALL_BUILD.vcxproj" (default target) (1) ->
"C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj" (default target) (4) ->
(ClCompile target) ->
/hip(1,1): error C1083: Cannot open source file: 'hip': No such file or directory [C:\Users\mbamg\AppData\Local\Temp\tmpd9u_s_jt\build\vendor\llama.cpp\ggml.vcxproj]

1 Warning(s)
1 Error(s)

Time Elapsed 00:00:02.54


*** CMake build failed
```

As with the original reporter, I've also tried setting CMake environment variables to force Clang compilation, with no change in result:

`[Environment]::SetEnvironmentVariable('CMAKE_ARGS', "-DLLAMA_HIPBLAS=on -DCMAKE_CXX_COMPILER='C:/Program Files/AMD/ROCm/5.7/bin/clang++.exe' -DCMAKE_C_ABI_COMPILED=FALSE -DCMAKE_CXX_ABI_COMPILED=FALSE -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_EXTENSIONS=OFF")`

`[Environment]::SetEnvironmentVariable('CMAKE_ARGS', "-DLLAMA_HIPBLAS=on -DCXX='C:/Program Files/AMD/ROCm/5.7/bin/clang++.exe' -DCMAKE_C_ABI_COMPILED=FALSE -DCMAKE_CXX_ABI_COMPILED=FALSE -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_EXTENSIONS=OFF")`

PS: Reading through #40 it's rather concerning that MSVC might be needed for Windows compilation. Is this still the case?

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.