lmstudio-ai / lmstudio-ai/mlx-engine

[Feature] MLX engine ignores built-in MTP (nextn) heads — native MTP speculative decoding missing on the MLX path

Open
#370 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
133
Avg merge
21h 6m
Merged PRs (30d)
1

Description

Summary

LM Studio 0.4.14 added MTP speculative decoding for the GGUF/llama.cpp path. The MLX engine has no equivalent: when a checkpoint ships built-in multi-token-prediction (nextn) heads, the MLX engine loads the model fine but does not use those heads, and gives no indication that it is skipping them.

On Apple Silicon this inverts the expected ranking. MLX has the better base decode, but ends up the slower option overall, purely because MTP exists only on the llama.cpp side.

Environment
  • MacBook Pro, M5 Max (40-core GPU), 128 GB unified memory, ~614 GB/s
  • LM Studio: current release as of 2026-08-20
  • Model: Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed (MLX, 19.85 GB, ships MTP heads)
  • Sampling: temp 0.6 / top_p 0.95 / top_k 20, full GPU
Observed

These are from normal chat use, not a controlled harness. I'm reporting them as a signal about the gap, not as a benchmark:

runtime MTP heads used decode
MTPLX (MLX runtime that uses the MTP heads, draft depth 2) yes 60-80 tok/s
LM Studio, MLX engine, same checkpoint no ~30 tok/s
LM Studio / Unsloth, GGUF, --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 yes 35-40 tok/s peak, 24-28 typical

Happy to re-measure any row under a configuration you specify, on a fixed prompt set — just say what you want measured and I'll run it.

Request
  1. Use the MTP / nextn heads in the MLX engine when a checkpoint provides them, with a configurable draft depth. Depth 2 was optimal on this machine; 1-4 covers the useful range.
  2. Independently of (1), and much cheaper: surface a load-time notice when a checkpoint contains MTP heads the engine cannot use. Today this is entirely silent, so there is no way for a user to tell that the MLX path is leaving speculation unused.
Related

#205 (MTP for GLM) and #323 (MTP model types fail to load) are both about architectures that don't load at all. This one is different: the model loads correctly, and the MTP heads inside it are ignored.

cc @neilmehta24 (assignee on #323)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Start by locating the MLX model-loading path and any existing MTP/nextn handling, then compare the related llama.cpp behavior. Done means configurable MLX draft depth uses available heads and a load-time notice appears when they cannot be used.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.