microsoft / microsoft/onnxruntime
TensorRT EP Resize with +inf input returns NaN while CPU/CUDA EP return +inf
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
`TensorrtExecutionProvider` produces `NaN` values for a minimal ONNX `Resize` model with `mode="linear"` when the input contains `+inf`, while `CPUExecutionProvider` and `CUDAExecutionProvider` both return `+inf` for the same model and input.
This is reproducible with a very small model:
- input shape: `[1, 1, 1, 2]`
- output shape: `[1, 1, 1, 4]`
- op: `Resize(mode="linear", coordinate_transformation_mode="half_pixel")`
- input values: `[+inf, 1.0]`
Observed outputs:
```text
CPUExecutionProvider: [inf, inf, inf, 1.0]
CUDAExecutionProvider: [inf, inf, inf, 1.0]
TensorrtExecutionProvider: [nan, nan, nan, 1.0]
```
The TensorRT EP profile indicates that the node was executed by TensorRT, not by CUDA/CPU fallback:
```text
trt_count=4
cuda_count=0
cpu_count=0
```
I could not find an existing issue for this specific TensorRT EP `Resize` + infinity mismatch.
### To reproduce
```python
import numpy as np
import onnx
import onnxruntime as ort
from pathlib import Path
from onnx import TensorProto, helper, numpy_helper
def const(name, arr):
return helper.make_node(
"Constant",
[],
[name],
value=numpy_helper.from_array(np.asarray(arr), name),
)
x = helper.make_tensor_value_info("x", TensorProto.FLOAT, [1, 1, 1, 2])
y = helper.make_tensor_value_info("y", TensorProto.FLOAT, [1, 1, 1, 4])
nodes = [
const("roi", np.asarray([], dtype=np.float32)),
const("scales", np.asarray([], dtype=np.float32)),
const("sizes", np.asarray([1, 1, 1, 4], dtype=np.int64)),
helper.make_node(
"Resize",
["x", "roi", "scales", "sizes"],
["y"],
mode="linear",
coordinate_transformation_mode="half_pixel",
nearest_mode="round_prefer_floor",
),
]
model = helper.make_model(
helper.make_graph(nodes, "resize_inf", [x], [y]),
opset_imports=[helper.make_opsetid("", 17)],
)
model.ir_version = 9
onnx.checker.check_model(model)
feed = {"x": np.asarray([np.inf, 1.0], dtype=np.float32).reshape(1, 1, 1, 2)}
for name, providers in [
("cpu", ["CPUExecutionProvider"]),
("cuda", ["CUDAExecutionProvider", "CPUExecutionProvider"]),
("trt", ["TensorrtExecutionProvider", "CUDAExecutionProvider", "CPUExecutionProvider"]),
]:
so = ort.SessionOptions()
so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
so.log_severity_level = 4
so.enable_profiling = True
sess = ort.InferenceSession(model.SerializeToString(), so, providers=providers)
out = sess.run(None, feed)[0]
profile = sess.end_profiling()
profile_text = Path(profile).read_text(errors="replace") if profile else ""
print(name, "providers=", sess.get_providers())
print(name, "output=", out.reshape(-1))
print(
name,
"profile counts:",
"trt=", profile_text.count("TensorrtExecutionProvider"),
"cuda=", profile_text.count("CUDAExecutionProvider"),
"cpu=", profile_text.count("CPUExecutionProvider"),
)
```
Output on my machine:
```text
cpu providers= ['CPUExecutionProvider']
cpu output= [inf inf inf 1.]
cpu profile counts: trt= 0 cuda= 0 cpu= 1
cuda providers= ['CUDAExecutionProvider', 'CPUExecutionProvider']
cuda output= [inf inf inf 1.]
cuda profile counts: trt= 0 cuda= 1 cpu= 0
trt providers= ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
trt output= [nan nan nan 1.]
trt profile counts: trt= 4 cuda= 0 cpu= 0
```
I also checked that ordinary finite inputs for the same model do not show this mismatch; the mismatch appears when the interpolation path contains `+inf`.
Note: I also reproduced that CPU EP and CUDA EP agree on the expected `[inf, inf, inf, 1.0]` result in onnxruntime-gpu 1.18.1. In my local TensorRT 8 setup, the 1.18.1 TensorRT provider library expected a newer TensorRT library and fell back to CUDA, so I could not independently validate the TensorRT EP path for 1.18.1 in this environment.
### Urgency
Not urgent.
### Platform
Linux
### OS Version
Ubuntu 20.04.5 LTS, Linux 5.4.0-100-generic
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
onnxruntime-gpu 1.17.1
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
TensorRT
### Execution Provider Library Version
TensorRT 8.6.1.6, CUDA 11.8, NVIDIA driver 580.105.08, NVIDIA GeForce RTX 3080 Ti; Python 3.10.20; onnx 1.17.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the supplied minimal Python reproducer with the listed CPU, CUDA, and TensorRT providers, then trace the TensorRT Execution Provider's Resize handling. Compare its interpolation behavior for +inf with the CPU and CUDA results; done means the TensorRT path no longer returns NaN for this model and the regression is covered by an appropriate test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100