microsoft / microsoft/onnxruntime

ORT CUDA Constant Folding Produces NaN from `Reciprocal(Sqrt(0)) * 0.0`

Open
#31,163 1 comment 0 reactions 0 assignees View on GitHub
ep:CUDA
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

The ORT CUDA optimizer constant-folds `Reciprocal(Sqrt(0)) = inf` using host
(IEEE 754) arithmetic, then folds `inf * 0.0 = NaN`. However, the actual CUDA
kernel for `Reciprocal` does **not** produce `inf` for input 0 — it returns a
very large finite value (~3.4e38). The unoptimized path is therefore **NaN-free**,
while the optimized path unexpectedly contains 208 NaN values.

### To reproduce

```python
from onnx import helper, TensorProto
import onnxruntime as ort
import numpy as np

zeros_t = helper.make_tensor("zeros_v", TensorProto.FLOAT, [13, 16, 16], [0.0] * 3328)
zero_t = helper.make_tensor("zero_v", TensorProto.FLOAT, [1, 1, 1, 1], [0.0])

nodes = [
helper.make_node('Constant', inputs=[], outputs=['v_z'], value=zeros_t),
helper.make_node('Sqrt', inputs=['v_z'], outputs=['v_sq']),
helper.make_node('Reciprocal', inputs=['v_sq'], outputs=['v_rc']),
helper.make_node('ReduceMin', inputs=['v_rc'], outputs=['v_rm'], axes=[-1], keepdims=0),
helper.make_node('Constant', inputs=[], outputs=['v_0'], value=zero_t),
helper.make_node('Mul', inputs=['v_rm', 'v_0'], outputs=['v_mul']),
]

v_mul_vi = helper.make_tensor_value_info('v_mul', TensorProto.FLOAT, [1, 1, 13, 16])
graph = helper.make_graph(nodes, 'minimal_nan_bug', [], [v_mul_vi])
model = helper.make_model(graph, opset_imports=[helper.make_opsetid('', 11)])

print("=== Model structure ===")
for i, n in enumerate(graph.node):
print(f" n[{i}] {n.op_type}: {list(n.input)} -> {list(n.output)}")

# With CUDA optimizer (default)
sess_opt = ort.InferenceSession(
model.SerializeToString(),
providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],
)

# Without optimizer (reference)
so_ref = ort.SessionOptions()
so_ref.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
sess_ref = ort.InferenceSession(
model.SerializeToString(),
providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],
sess_options=so_ref,
)

opt = np.asarray(sess_opt.run(None, {})[0])
ref = np.asarray(sess_ref.run(None, {})[0])

print(f"opt (optimizer on): shape={opt.shape}, nan_count={np.isnan(opt).sum()}, values={opt.flatten()[:8]}")
print(f"ref (optimizer off): shape={ref.shape}, nan_count={np.isnan(ref).sum()}, values={ref.flatten()[:8]}")

```

### Urgency

No. I found this by fuzzing testing.

### Platform

Linux

### OS Version

5.4.0-162-generic

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.27.0

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

CUDA

### Execution Provider Library Version

CUDA 13.0, Driver 580.76.05, GPU: NVIDIA GeForce RTX 3080 Ti

Contributor guide

Open the contributing guide

Research direction

Start with the CUDA optimizer's constant-folding path and the Reciprocal, Sqrt, ReduceMin, and Mul nodes in the reproduction. Run the supplied Python model with optimization enabled and disabled, then trace the CUDA and host results. Done means the optimized and unoptimized paths agree without introducing NaN values.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend, machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.