microsoft / microsoft/onnxruntime
[Feature Request] Make TensorRT verbose when passing `log_severity_level = 0`
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Currently, even if we pass `log_severity_level = 0`, TensorRT is not verbose.
I'd like to see logs as:
```
[12/30/2022-16:53:21] [V] [TRT] Parsing node: /transformer/h.14/attn/Shape_9 [Shape]
[12/30/2022-16:53:21] [V] [TRT] Searching for input: /transformer/h.14/attn/Slice_output_0
[12/30/2022-16:53:21] [V] [TRT] /transformer/h.14/attn/Shape_9 [Shape] inputs: [/transformer/h.14/attn/Slice_output_0 -> (-1, -1, 16, 64)[FLOAT]],
[12/30/2022-16:53:21] [V] [TRT] Registering layer: /transformer/h.14/attn/Shape_9 for ONNX node: /transformer/h.14/attn/Shape_9
[12/30/2022-16:53:21] [V] [TRT] Registering tensor: /transformer/h.14/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.14/attn/Shape_9_output_0
```
when I pass `log_severity_level = 0`, but I'm not seeing them.
Reproduce:
```python
import onnxruntime as ort
session_options = ort.SessionOptions()
session_options.log_severity_level = 0
session = ort.InferenceSession(
"/home/user/scripts/gptj_onnx/decoder_model.onnx"
providers=["TensorrtExecutionProvider"],
sess_options=session_options
)
```
### Describe scenario use case
It would help for debugging.
For example, I had a case when the sole log printing was `external/onnx-tensorrt/onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped`, and you would get stuck there for several minutes not knowing what is going on, while trtexec is more verbose and intertwines the logs. Then, you can get stuck at `[I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution` with no more indication of what is happening (imo is engine build).
Contributor guide
Assessment
This issue has not been assessed yet.