huggingface / huggingface/candle
[feature] Add DeepSeek-V3, Llama 4 and Qwen3-MoE model architectures
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
### Motivation
`candle-transformers` already covers a lot (llama, qwen2/qwen3, deepseek2,
gemma2/3/4, glm4, mixtral, plus VLMs), but a few recent dense/MoE families that
users frequently ask for are still missing.
### Gap in candle (please confirm)
- **DeepSeek-V3** — only `deepseek2` (V2) is present; V3 adds aux-loss-free MoE
routing, Multi-Token Prediction and FP8.
- **Llama 4** — MoE + interleaved/iRoPE; not present.
- **Qwen3-MoE** — only dense `qwen3` appears to be present.
### Proposed implementation
- New modules `candle-transformers/src/models/{deepseek3,llama4,qwen3_moe}.rs`,
following the existing MoE models (`mixtral`, `deepseek2`).
- HF `config.json` parsing + `VarBuilder` loading + quantized variant where it
makes sense.
- A minimal `candle-examples/examples/` per family for validation.
### Use-case / example
```rust
// Load Qwen3-MoE like any other candle-transformers model.
let cfg: Qwen3MoeConfig = serde_json::from_slice(&std::fs::read("config.json")?)?;
let model = Qwen3Moe::load(vb, &cfg)?;
let logits = model.forward(&input_ids, /*pos=*/0)?;
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the existing candle-transformers models, especially mixtral and deepseek2, and confirm the stated gaps for DeepSeek-V3, Llama 4, and Qwen3-MoE. The proposed entry points are candle-transformers/src/models/{deepseek3,llama4,qwen3_moe}.rs, with matching candle-examples/examples/ validation examples. Done means the three families support config parsing, VarBuilder loading, and appropriate quantized variants.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100