[Reproducibility] dsr1 0528 dynamo + mtp on b200 / [Reproducibility] DSR1 0528 Dynamo + MTP 在 B200 上的复现

Open
#1,691 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Quiet
Tech stack
kubernetes
Domain
distributed-systems, infrastructure, machine-learning, performance

Research direction

Start by comparing the Kubernetes deployment with recipes/trtllm/b200-fp4/1k1k/mtp/ctx3_gen5_dep4_batch512_eplb0_mtp1.yaml and the configuration described in GitHub Actions run 21484975323. Check the reported frontend deployment and performance settings against the available run details. Done means documenting the missing configuration or explaining why the reproduced throughput differs from 10,012 tok/s per GPU.

Written by the indexing model from the issue text.

Description

Hey,

I've been trying to reproduce the results for the following run:
ISL = 1k
OSK = 1k

B200 (Dynamo TRT, MTP)
Date: 2026-01-29
Image: nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.8.1.post1
Interactivity (tok/s/user): 21.338126345946833
Output Token Throughput per GPU (tok/s/gpu): 10,012.214
Total GPUs: 32
Prefill: 12 GPUs, TP: 4, EP: 4, DPA: True, Workers: 3
Decode: 20 GPUs, TP: 4, EP: 4, DPA: True, Workers: 5
Concurrency: 10860
Precision: FP4
GitHub Actions Run

I have 4 nodes of B200 sxm, i am using K8s to deploy the same configuration as you did here:
https://github.com/NVIDIA/srt-slurm/blob/sa-submission-q2-2026/recipes/trtllm/b200-fp4/1k1k/mtp/ctx3_gen5_dep4_batch512_eplb0_mtp1.yaml

No matter what I did, my results still fall under 10K TPS per GPU. Current best result is ~8.3K per decode gpu.

I have validated the kv transfer is via gpu direct.

The logs of the run already expired and therefore, ask if there is a way to get them or at least share more about how to be able to reproduce the results , e.g, how many frontends were deployed ? or were the system was configured to performance?

Thanks

中文说明

用户尝试在 4 节点 B200 SXM（K8s 部署）上复现 DSR1 0528 使用 Dynamo TRT + MTP 的基准测试结果（ISL=1k, OSL=1k, FP4, 32 GPU 分离式推理），但每 GPU 输出吞吐量仅达到约 8.3K tok/s，低于发布的 10,012 tok/s。已确认 KV 传输使用 GPU Direct。由于 GitHub Actions 运行日志已过期，请求提供更多复现信息，例如前端部署数量和系统性能配置。

Dominant language
Python
Stars
1.7k
Forks
303
Avg merge
1d 13h
Merged PRs (30d)
284

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from SemiAnalysisAI/InferenceX

All issues in SemiAnalysisAI/InferenceX

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.