[AutoDeploy]: Investigate SuperV3 with MTP not scaling well
Open
Nobody has claimed this yet.
AutoDeploy
feature request
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
🚀 The feature, motivation and pitch
AD surpasses PT at WS=1 across all concurrencies — 18% faster at c=1, near-parity (1%) at c=64.
AD fails to scale at WS=4: PT leads by 20–35%. The gap is driven by poor WS=1→4 scaling on the AD side: PT scales 1.52× from WS=1 to WS=4 at c=1 (4.67ms→3.07ms), while AD only gains 1.04× (3.82ms→3.67ms).
Investigate and resolve the poor scaling on AutoDeploy side.
Baseline branch: nv-auto-deploy:gagam/super-mtp-perf-2-replay (see #13725)
Alternatives
No response
Additional context
See SuperV3 MTP ticket #12359
Scripts, configs and experiment data:
https://gitlab-master.nvidia.com/ghubaraagam/agent-reports/-/tree/main/260428_superv3_mtp?ref_type=heads
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.