kvcache-ai / kvcache-ai/ktransformers

Support DSpark speculative decoding for DeepSeek-V4-Flash-0731 in sglang-kt

Open
#2,118 1 comment 3 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### Reminder

- [x] I have read the above rules and searched the existing issues.

### Description

## Summary

DeepSeek-V4-Flash-0731 (official release, 2026-07-31) ships a new **DSpark** speculative decoding module instead of the NextN-style MTP head used in DeepSeek-V4-Flash-Preview. `sglang-kt==0.6.4` only supports `EAGLE`/`NEXTN` speculative algorithms and cannot load the new draft head, forcing users to disable speculative decoding entirely (~2x decode speed loss).

Request: port DSPARK support to sglang-kt (upstream SGLang supports it since v0.5.16).

## Background

Per the official release notes, DeepSeek-V4-Flash-0731 "has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached."

The checkpoint layout changed:

- **Preview**: single NextN MTP layer with top-level keys `model.enorm/hnorm/e_proj/h_proj` (+ `weight_scale_inv`)
- **0731**: three full MoE draft layers `mtp.0`, `mtp.1`, `mtp.2` (4,705 `mtp.*` keys in `model.safetensors.index.json`), each containing attention + 256 routed experts + 1 shared expert, plus `mtp.2.confidence_head.proj` and `mtp.2.markov_head.markov_w1/w2`

New `config.json` fields:

```json
"dspark_block_size": 5,
"dspark_noise_token_id": 128799,
"dspark_target_layer_ids": [40, 41, 42],
"dspark_markov_rank": 256

### Pull Request

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by comparing the new config.json fields and the 4,705 mtp.* entries in model.safetensors.index.json with the upstream SGLang v0.5.16 DSpark support. Trace how sglang-kt currently handles EAGLE/NEXTN, then verify that DeepSeek-V4-Flash-0731 loads its three draft layers and performs speculative decoding without disabling the feature.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.