abetlen / abetlen/llama-cpp-python
Workflow Example for CUDA 12.6 Windows
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.4k
- PR merge metrics
- PR metrics pending
Description
For those trying to use GitHub Actions to build the latest version (v0.3.7) with CUDA 12.6 for Windows but failed, maybe installing VS version >=17.12 and CUDA directly like [here](https://github.com/Jimver/cuda-toolkit?tab=readme-ov-file#example-usage) can solve the issue, [here](https://github.com/lyxkilo/llama-cpp-python/blob/main/.github/workflows/build-wheels-cuda.yaml) is an example workflow.
P.S. change the value of CMAKE argument "DCMAKE_CUDA_ARCHITECTURES" to your local CUDA arch instead of "all" in the original workflow can reduce a lot of compilation time.
https://cmake.org/cmake/help/latest/prop_tgt/CUDA_ARCHITECTURES.html
https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/#gpu-feature-list
Contributor guide
Assessment
This issue has not been assessed yet.