huggingface / huggingface/candle

[feature] Add DeepSeek-V3, Llama 4 and Qwen3-MoE model architectures

Open
#3,653 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

### Motivation
`candle-transformers` already covers a lot (llama, qwen2/qwen3, deepseek2,
gemma2/3/4, glm4, mixtral, plus VLMs), but a few recent dense/MoE families that
users frequently ask for are still missing.

### Gap in candle (please confirm)
- **DeepSeek-V3** — only `deepseek2` (V2) is present; V3 adds aux-loss-free MoE
routing, Multi-Token Prediction and FP8.
- **Llama 4** — MoE + interleaved/iRoPE; not present.
- **Qwen3-MoE** — only dense `qwen3` appears to be present.

### Proposed implementation
- New modules `candle-transformers/src/models/{deepseek3,llama4,qwen3_moe}.rs`,
following the existing MoE models (`mixtral`, `deepseek2`).
- HF `config.json` parsing + `VarBuilder` loading + quantized variant where it
makes sense.
- A minimal `candle-examples/examples/` per family for validation.

### Use-case / example
```rust
// Load Qwen3-MoE like any other candle-transformers model.
let cfg: Qwen3MoeConfig = serde_json::from_slice(&std::fs::read("config.json")?)?;
let model = Qwen3Moe::load(vb, &cfg)?;
let logits = model.forward(&input_ids, /*pos=*/0)?;
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the existing candle-transformers models, especially mixtral and deepseek2, and confirm the stated gaps for DeepSeek-V3, Llama 4, and Qwen3-MoE. The proposed entry points are candle-transformers/src/models/{deepseek3,llama4,qwen3_moe}.rs, with matching candle-examples/examples/ validation examples. Done means the three families support config parsing, VarBuilder loading, and appropriate quantized variants.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.