NVIDIA / NVIDIA/TensorRT-LLM

[Feature Request] AutoDeploy: enable DeepSeek R1 MTP

Open
#15,277 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

AutoDeploy Speculative Decoding
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

Summary

Enable AutoDeploy support for DeepSeek R1 MTP / one-model speculative decoding.

Motivation

DeepSeek R1 exposes MTP layers in the DeepSeek-V3-family checkpoint layout, but AutoDeploy currently does not have full DeepSeek-V3-family MTP/Eagle drafter support. This blocks reduced smoke coverage and eventual real-weight validation for DeepSeek R1 MTP through AutoDeploy.

Requested Support

  • Register DeepSeek-V3-family MTP with AutoDeploy's Eagle/MTP drafter path.
  • Support DeepSeek R1 MTP checkpoint remapping from target model layer indices into the drafter.
  • Keep shared target embedding and lm_head behavior aligned with the PyTorch backend.
  • Preserve realistic quantization behavior, including not graph-quantizing shared target modules that are called outside the exported graph.

Initial Validation

The first step is reduced smoke coverage because the full DeepSeek R1 model cannot be run in this local environment. The smoke should verify that the AutoDeploy MTP path initializes, loads the reduced checkpoint layout, compiles, and runs end to end.

Follow-Up Accuracy Validation

Add a full-model accuracy test in test_llm_api_autodeploy.py for DeepSeek R1 with MTP enabled. The test should verify that enabling MTP does not degrade accuracy relative to the non-MTP AutoDeploy path, and it should enforce a minimum acceptance-rate threshold so we know the drafter is actually useful rather than merely functional.

Acceptance Criteria

  • A reduced-layer DeepSeek R1 + MTP AutoDeploy smoke test runs end to end.
  • A no-MTP reduced DeepSeek R1 AutoDeploy smoke remains passing.
  • Quantized DeepSeek R1 configs keep lm_head excluded when required by shared-head MTP execution.
  • A full-model AutoDeploy accuracy test covers DeepSeek R1 MTP in test_llm_api_autodeploy.py.
  • The accuracy test checks no accuracy degradation with MTP and enforces a minimum acceptance rate.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing AutoDeploy Eagle/MTP drafter path and DeepSeek-V3-family checkpoint handling; the issue does not name the implementation files, so trace those entry points before changing code. Use reduced smoke coverage first, then update test_llm_api_autodeploy.py for full-model accuracy. Done means both reduced paths pass, shared-head quantization is safe, and MTP meets the accuracy and acceptance-rate criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.