NVIDIA-NeMo/Export-Deploy

View on GitHub

A library for exporting models including NeMo and Hugging Face to optimized inference backends, and deploying them for efficient querying

Stars: 42
Forks: 18
Open beginner issues: 0
Indexed issues: 28
Avg merge: 1d 2h
Merged PRs (30d): 8

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 10, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

28 open issues indexed

[ALERT] Access to NVIDIA's Self-Hosted Runners is Expiring Open

nvidia-runners

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

NVIDIA-NeMo/Export-Deploy#759 · Opened Aug 30, 2026
Update ED to use Dynamic Inference Engine from MCore Open

NVIDIA-NeMo/Export-Deploy#693 · 1 assignee · Opened May 22, 2026
Incorrect handling of reasoning traces for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 in MBridge format Open

bug

NVIDIA-NeMo/Export-Deploy#682 · 1 comment · 1 assignee · Opened May 22, 2026
Server returns empty answer when too many tokens requested Open

bug

NVIDIA-NeMo/Export-Deploy#652 · 1 assignee · Opened Apr 13, 2026
VLM endpoint fails when there are no images in the request Open

bug

NVIDIA-NeMo/Export-Deploy#632 · 1 assignee · Opened Mar 12, 2026
Ray Deployment of moonlight 16B (MBridge) fails Open

bug

NVIDIA-NeMo/Export-Deploy#616 · 1 assignee · Opened Feb 23, 2026
Stop word support for mcore inference Open

enhancement

NVIDIA-NeMo/Export-Deploy#564 · 1 comment · 1 assignee · Opened Jan 20, 2026
Copy changes in PR 501 and 505 to scripts under new llm dir Open

bug

NVIDIA-NeMo/Export-Deploy#504 · 1 assignee · Opened Nov 6, 2025
Export and Deploy PTQed Megatron-Bridge Models Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Export-Deploy#457 · 1 comment · Opened Oct 17, 2025
Create separate scripts for all the deployment paths Open

enhancement

NVIDIA-NeMo/Export-Deploy#438 · 1 assignee · Opened Oct 6, 2025
Check params like use_greedy and repetition_penalty for NemoQueryLLMPyTorch class Open

bug

NVIDIA-NeMo/Export-Deploy#434 · 1 assignee · Opened Oct 4, 2025
Stopword support for in-framework deployment Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA-NeMo/Export-Deploy#430 · 2 comments · Opened Oct 1, 2025
Go over the docs and update the support matrices Open

documentation

NVIDIA-NeMo/Export-Deploy#425 · 1 assignee · Opened Oct 1, 2025
Move operations like chat templates to the server side for VLM deployment Open

enhancement

NVIDIA-NeMo/Export-Deploy#423 · 1 comment · 1 assignee · Opened Oct 1, 2025
Introduce consistent and intuitive parameter naming for NeMo TensorRTLLM class vs TensorRT-LLM library, and across all export and deploy options Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Export-Deploy#412 · Opened Sep 24, 2025
Refactor tokenizer handling in nemo.export module Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Export-Deploy#411 · Opened Sep 24, 2025
Integrate exporting & deploying modelopt (quantized) unified checkpoints Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA-NeMo/Export-Deploy#410 · 1 comment · Opened Sep 24, 2025
Support exporting fp4 models Open

enhancement

NVIDIA-NeMo/Export-Deploy#409 · 1 assignee · Opened Sep 24, 2025
Add multi batch size cuda graphs support for M-Bridge in-framework deployment. Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA-NeMo/Export-Deploy#408 · Opened Sep 24, 2025
Add GenAI-Perf Benchmark tool Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 30/100

NVIDIA-NeMo/Export-Deploy#407 · Opened Sep 24, 2025
Support quantized m-bridge checkpoint for vLLM export Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Export-Deploy#406 · Opened Sep 24, 2025
Find the fastest inference options among vLLM, TensorRT-LLM, and M-Bridge Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Export-Deploy#405 · 1 comment · Opened Sep 24, 2025
Quantize M-Bridge and HF checkpoints during In-Framework Deployment Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Export-Deploy#404 · Opened Sep 24, 2025
Export and Deploy Tutorials Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Export-Deploy#402 · Opened Sep 24, 2025
VLM in-framework support deployment using M-Bridge checkpoint Open

enhancement

NVIDIA-NeMo/Export-Deploy#401 · 1 assignee · Opened Sep 24, 2025
vLLM export and quantization support for NeMo RL Open

enhancement

NVIDIA-NeMo/Export-Deploy#399 · 1 assignee · Opened Sep 24, 2025
Support M-Bridge checkpoints for TensorRT-LLM Export Open

enhancement

NVIDIA-NeMo/Export-Deploy#398 · 1 comment · 1 assignee · Opened Sep 24, 2025
NeMo-Run support for Ray based multi-node deployment Open

enhancement

NVIDIA-NeMo/Export-Deploy#397 · 1 assignee · Opened Sep 24, 2025

NVIDIA-NeMo/Export-Deploy

Get new issues in your inbox