microsoft / microsoft/onnxruntime

[Feature Request] Make TensorRT verbose when passing `log_severity_level = 0`

Open
#14,122 1 comment 1 reaction 1 assignee Claimed by @yf711 View on GitHub
ep:TensorRT feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Currently, even if we pass `log_severity_level = 0`, TensorRT is not verbose.

I'd like to see logs as:

```
[12/30/2022-16:53:21] [V] [TRT] Parsing node: /transformer/h.14/attn/Shape_9 [Shape]
[12/30/2022-16:53:21] [V] [TRT] Searching for input: /transformer/h.14/attn/Slice_output_0
[12/30/2022-16:53:21] [V] [TRT] /transformer/h.14/attn/Shape_9 [Shape] inputs: [/transformer/h.14/attn/Slice_output_0 -> (-1, -1, 16, 64)[FLOAT]],
[12/30/2022-16:53:21] [V] [TRT] Registering layer: /transformer/h.14/attn/Shape_9 for ONNX node: /transformer/h.14/attn/Shape_9
[12/30/2022-16:53:21] [V] [TRT] Registering tensor: /transformer/h.14/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.14/attn/Shape_9_output_0
```

when I pass `log_severity_level = 0`, but I'm not seeing them.

Reproduce:

```python
import onnxruntime as ort

session_options = ort.SessionOptions()
session_options.log_severity_level = 0

session = ort.InferenceSession(
"/home/user/scripts/gptj_onnx/decoder_model.onnx"
providers=["TensorrtExecutionProvider"],
sess_options=session_options
)
```

### Describe scenario use case

It would help for debugging.

For example, I had a case when the sole log printing was `external/onnx-tensorrt/onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped`, and you would get stuck there for several minutes not knowing what is going on, while trtexec is more verbose and intertwines the logs. Then, you can get stuck at `[I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution` with no more indication of what is happening (imo is engine build).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.