microsoft / microsoft/onnxruntime

[Feature Request] Support profilingVerbosity option in TensorRT Execution Provider

Open
#28,582 0 comments 0 reactions 0 assignees View on GitHub
ep:TensorRT feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Currently, the TensorRT Execution Provider in ONNX Runtime does not expose an option to configure TensorRT's `ProfilingVerbosity` (such as `kDETAILED`, `kLAYER_NAMES_ONLY`, or `kNONE`).

I would like to request adding a new provider option (e.g., `trt_profiling_verbosity`) to the TensorRT EP. This option would map to TensorRT's `IBuilderConfig::setProfilingVerbosity()` API during the engine building phase.

Ideally, it would accept string values corresponding to the TensorRT enums, for example:

* `"detailed"` -> `nvinfer1::ProfilingVerbosity::kDETAILED`
* `"layer_names_only"` -> `nvinfer1::ProfilingVerbosity::kLAYER_NAMES_ONLY`
* `"none"` -> `nvinfer1::ProfilingVerbosity::kNONE`

### Describe scenario use case

When optimizing deep learning models in ONNX Runtime using the TensorRT EP, developers frequently rely on profiling tools like NVIDIA Nsight Systems (nsys) to identify performance bottlenecks.

By default, without detailed profiling verbosity, the generated engines lack granular visibility into specific layer names, kernel dimensions, and operation types. Currently, to obtain this level of detailed metadata, developers are forced to leave the ORT environment and use standalone CLI tools like `trtexec` on the exported model.

If users can set `trt_profiling_verbosity` to `detailed` directly via ORT's provider options, it would drastically improve the workflow. It allows developers to check detailed layer-level metrics seamlessly at the ONNX Runtime TensorRT EP level, eliminating the hassle of using external tools like `trtexec`. This keeps the entire debugging and performance tuning process highly convenient and contained within a single pipeline.

Contributor guide

Open the contributing guide

Research direction

Start by locating the TensorRT Execution Provider's provider-option handling and the engine-building phase where TensorRT's IBuilderConfig is available. Verify how the option should accept detailed, layer_names_only, and none, map them to the corresponding ProfilingVerbosity values, and confirm the behavior through the relevant TensorRT EP tests if present.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.