huggingface / huggingface/candle
Vulkan support
- Dominant language
- Rust
- Stars
- 21.1k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
The m2 metal support is great for people running macs, however it doesn't cover the majority of devices out there. Supporting the Vulkan graphics library can greatly increase the reach of GPUs that could be used for inference.
On a related note, [llama.cpp recently landed support for it](https://github.com/ggerganov/llama.cpp/commit/2307523d322af762ae06648b29ec5a9eb1c73032) and tools like Ollama are working on integrating this support.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing Candle's existing M2 Metal support and the linked llama.cpp Vulkan implementation to understand the scope of a Vulkan backend. The issue is complete when Vulkan support enables inference on a broader range of GPUs, with the relevant behavior validated across supported devices.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- computer-graphics, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100