lmstudio-ai / lmstudio-ai/mlx-engine
[Feature] MLX engine ignores built-in MTP (nextn) heads — native MTP speculative decoding missing on the MLX path
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
Summary
LM Studio 0.4.14 added MTP speculative decoding for the GGUF/llama.cpp path. The MLX engine has no equivalent: when a checkpoint ships built-in multi-token-prediction (nextn) heads, the MLX engine loads the model fine but does not use those heads, and gives no indication that it is skipping them.
On Apple Silicon this inverts the expected ranking. MLX has the better base decode, but ends up the slower option overall, purely because MTP exists only on the llama.cpp side.
Environment
- MacBook Pro, M5 Max (40-core GPU), 128 GB unified memory, ~614 GB/s
- LM Studio: current release as of 2026-08-20
- Model:
Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed(MLX, 19.85 GB, ships MTP heads) - Sampling: temp 0.6 / top_p 0.95 / top_k 20, full GPU
Observed
These are from normal chat use, not a controlled harness. I'm reporting them as a signal about the gap, not as a benchmark:
| runtime | MTP heads used | decode |
|---|---|---|
| MTPLX (MLX runtime that uses the MTP heads, draft depth 2) | yes | 60-80 tok/s |
| LM Studio, MLX engine, same checkpoint | no | ~30 tok/s |
LM Studio / Unsloth, GGUF, --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 |
yes | 35-40 tok/s peak, 24-28 typical |
Happy to re-measure any row under a configuration you specify, on a fixed prompt set — just say what you want measured and I'll run it.
Request
- Use the MTP /
nextnheads in the MLX engine when a checkpoint provides them, with a configurable draft depth. Depth 2 was optimal on this machine; 1-4 covers the useful range. - Independently of (1), and much cheaper: surface a load-time notice when a checkpoint contains MTP heads the engine cannot use. Today this is entirely silent, so there is no way for a user to tell that the MLX path is leaving speculation unused.
Related
#205 (MTP for GLM) and #323 (MTP model types fail to load) are both about architectures that don't load at all. This one is different: the model loads correctly, and the MTP heads inside it are ignored.
cc @neilmehta24 (assignee on #323)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by locating the MLX model-loading path and any existing MTP/nextn handling, then compare the related llama.cpp behavior. Done means configurable MLX draft depth uses available heads and a load-time notice appears when they cannot be used.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100