huggingface / huggingface/candle
Added support for Vulkan, any interest?
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
I have a Intel Arc A770 16GB GPU and wanted to use it with candle.
I took niklasha's work on niklas-vulkan-2 branch cherry-pick's into the current main branch.
I (when I say I, I mean I was the navigator, Codex 5.2 max did the work) added the following:
Added Vulkan queue-family selection and synchronize() so VulkanDevice uses compute-capable queues and can block on GPU work (device.rs).
Expanded Vulkan storage surface with raw_buffer() access for kernel dispatch and fixed error wiring (storage.rs).
Wired Vulkan kernel registry to include matmul, norms, softmax, masked softmax, and quantized kernels (lib.rs).
Added F32/F16 matmul shader stubs and norm/softmax shaders for initial Vulkan ops coverage (*.comp).
Implemented Vulkan masked softmax and staged SDPA path with GQA support in candle-nn (ops.rs).
Added Vulkan smoke tests and masked softmax correctness test (vulkan_smoke_tests.rs, vulkan_masked_softmax.rs).
Fixed missing imports and push-constant binding for Vulkan command execution (storage.rs).
Added bytemuck + vulkano-shaders feature wiring for Vulkan builds (Cargo.toml).
Introduced QVulkanStorage backed by raw byte buffers with dequantize/quantize helpers (vulkan.rs).
Added Vulkan quantized matmul kernels for Q5_0 and Q8_0 (naive, F32 output) (qmatmul_q5_0_f32.comp, qmatmul_q8_0_f32.comp).
Hooked Vulkan quantized path into QTensor forward and added Vulkan quantized tests (mod.rs, vulkan_quantized_tests.rs).
Added a dequantize‑fallback backward path for QLoRA-style gradients (mod.rs).
Cleaned up dummy Vulkan stubs to match new quantized API surface (dummy_vulkan.rs).
Fixed multiple test harness macro/feature mismatches to compile with Vulkan enabled (test_utils.rs, *.rs).
Contributor guide
No contributing guide indexed for this repository
Research direction
Review the Vulkan changes listed across device.rs, storage.rs, lib.rs, ops.rs, vulkan.rs, mod.rs, and the named shader and test files to understand the proposed scope. Run the Vulkan smoke, masked-softmax, and quantized tests with Vulkan enabled, then determine whether this broad implementation is wanted and what maintainers consider complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100