NVIDIA / NVIDIA/TensorRT-LLM

[Bug]: AutoDeploy model-coverage hits NVRTC compilation failed across multiple models

Open
#14,676 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

AutoDeploy bug Customized kernels
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

System Info
  • CPU architecture: x86_64
  • GPU properties: not captured from CI artifact
  • Libraries
    • TensorRT-LLM branch or tag: main (pipeline context)
    • Container used: AutoDeploy model-coverage CI container
  • OS: CI environment
  • Additional context: terminal pipeline 52785950
Who can help?

No response

Information
  • The official example scripts
  • My own modified scripts
Tasks
  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)
Reproduction

Pipeline 52785720 resolved to terminal model-coverage pipeline 52785950.

Affected registry entries currently disabled in examples/auto_deploy/model_registry/models.yaml include:

  • zai-org/GLM-5-FP8
  • MiniMaxAI/MiniMax-M2.5
  • MiniMaxAI/MiniMax-M2.7
  • deepseek-ai/DeepSeek-V3.2
  • deepseek-ai/DeepSeek-V3.2-Speciale
  • MiniMaxAI/MiniMax-M2
  • deepseek-ai/DeepSeek-Prover-V2-671B
  • deepseek-ai/DeepSeek-V3
  • Qwen/Qwen3-0.6B-FP8

Representative evidence from per-test stdout logs ends with:

RuntimeError: NVRTC compilation failed

One concrete example is MiniMaxAI/MiniMax-M2.7, where the traceback tail runs through tensorrt_llm/_torch/auto_deploy/custom_ops/quantization/trtllm_quant.py into torch.ops.trtllm.fp8_block_scaling_gemm(...) before failing with RuntimeError: NVRTC compilation failed.

Expected behavior

These model-coverage entries should either compile and run successfully, or fail with a more specific unsupported-configuration signal if FP8 block-scaling GEMM is not supported in the current path.

actual behavior

TensorRT-LLM fails during AutoDeploy execution with RuntimeError: NVRTC compilation failed across multiple distinct models, suggesting a shared runtime or codegen failure rather than a model-specific problem.

additional notes

This looks like a TensorRT-LLM / AutoDeploy runtime issue, not a Hugging Face access problem. The same failure string appears across nine disabled entries, which makes it a good candidate for one general issue

Before submitting a new issue...
  • Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with terminal pipeline 52785950 and the disabled entries in examples/auto_deploy/model_registry/models.yaml. Trace the MiniMaxAI/MiniMax-M2.7 failure through tensorrt_llm/_torch/auto_deploy/custom_ops/quantization/trtllm_quant.py and the fp8_block_scaling_gemm operation. Done means the affected entries run successfully or report a specific unsupported-configuration signal instead of the generic NVRTC compilation failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.