kvcache-ai / kvcache-ai/ktransformers
Support DSpark speculative decoding for DeepSeek-V4-Flash-0731 in sglang-kt
- Dominant language
- Python
- Stars
- 19.5k
- Forks
- 1.6k
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 27
Description
### Reminder
- [x] I have read the above rules and searched the existing issues.
### Description
## Summary
DeepSeek-V4-Flash-0731 (official release, 2026-07-31) ships a new **DSpark** speculative decoding module instead of the NextN-style MTP head used in DeepSeek-V4-Flash-Preview. `sglang-kt==0.6.4` only supports `EAGLE`/`NEXTN` speculative algorithms and cannot load the new draft head, forcing users to disable speculative decoding entirely (~2x decode speed loss).
Request: port DSPARK support to sglang-kt (upstream SGLang supports it since v0.5.16).
## Background
Per the official release notes, DeepSeek-V4-Flash-0731 "has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached."
The checkpoint layout changed:
- **Preview**: single NextN MTP layer with top-level keys `model.enorm/hnorm/e_proj/h_proj` (+ `weight_scale_inv`)
- **0731**: three full MoE draft layers `mtp.0`, `mtp.1`, `mtp.2` (4,705 `mtp.*` keys in `model.safetensors.index.json`), each containing attention + 256 routed experts + 1 shared expert, plus `mtp.2.confidence_head.proj` and `mtp.2.markov_head.markov_w1/w2`
New `config.json` fields:
```json
"dspark_block_size": 5,
"dspark_noise_token_id": 128799,
"dspark_target_layer_ids": [40, 41, 42],
"dspark_markov_rank": 256
### Pull Request
_No response_
Contributor guide
Research direction
Start by comparing the new config.json fields and the 4,705 mtp.* entries in model.safetensors.index.json with the upstream SGLang v0.5.16 DSpark support. Trace how sglang-kt currently handles EAGLE/NEXTN, then verify that DeepSeek-V4-Flash-0731 loads its three draft layers and performs speculative decoding without disabling the feature.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100