NVIDIA / NVIDIA/TensorRT-Model-Connect

[Docs] Add TensorRT Model Connect to the TensorRT documentation

Open
#1,100 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Platform
Dominant language
Python
Stars
254
Forks
58
Avg merge
1d 7h
Merged PRs (30d)
235

Description

Summary

Add TensorRT Model Connect to the main TensorRT documentation so users can discover the simplest path from a supported PyTorch model to end-to-end TensorRT inference without already knowing the project name.

The documentation should explain when Model Connect is the appropriate entry point and lead users directly to a working quickstart.

Content

  • Explain what TensorRT Model Connect provides and who should use it.
  • Include the two-command build-and-run workflow:
trtmc build Qwen/Qwen3-0.6B --max-cache-length 16384 --output qwen3-0.6b.bundle
trtmc run ./qwen3-0.6b.bundle --prompt "What is the capital of France? Answer in one word." --chat-template --no-thinking
# Generated text: Paris
  • Explain the relationship between the three layers:
    • TensorRT provides the underlying inference optimizer and runtime.
    • TensorRT Model Connect provides model-level reference implementations, bundle generation, and task-oriented native runtime workflows.
    • TensorRT Edge-LLM is the production-focused path for qualified LLM and VLM deployment on supported NVIDIA edge platforms.
  • Link to:
  • Clearly distinguish qualified support for an exact checkpoint and configuration from best-effort compatibility with related checkpoints or configurations.
  • Document the relevant checkpoint revision, precision, platform, and runtime-version boundaries where support claims are made.
  • Add Model Connect to the relevant navigation, ecosystem, and getting-started surfaces in the main TensorRT documentation.
  • Ensure the TensorRT documentation directs eligible PyTorch-model users to the Model Connect quickstart.

Acceptance criteria

A user entering through the TensorRT documentation can:

  • Understand what TensorRT Model Connect provides.
  • Decide when to use Model Connect, TensorRT Edge-LLM, or lower-level TensorRT APIs.
  • Reach and complete a working Model Connect quickstart without already knowing the project name.
  • Distinguish verified configurations from best-effort compatibility.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the main TensorRT documentation, its navigation, ecosystem, and getting-started surfaces, then review the linked Model Connect quickstart, setup overview, supported-models page, tutorials, and repository. Add the requested Model Connect guidance, commands, links, support boundaries, and routing for eligible PyTorch users; done means a reader can choose the appropriate TensorRT path and complete the quickstart.

Written by the indexing model from the issue text.

Assessment

Tech stack
pytorch
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.