microsoft / microsoft/onnxruntime
CUDA ArgMax on all -inf returns INT32_MAX sentinel value
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
The CUDAExecutionProvider's ArgMax kernel returns `INT32_MAX` (2147483647) when ALL
elements along the reduction axis are `-inf`. The optimizer constant-folds this case
correctly (returns 0), but the runtime CUDA kernel produces an out-of-range sentinel.
### To reproduce
```python
from onnx import helper, TensorProto
import onnxruntime as ort
import numpy as np
SHAPE = [4, 12] # any [N, M] where M >= 2
v_t = helper.make_tensor("v_t", TensorProto.FLOAT, SHAPE,
[float('-inf')] * (SHAPE[0] * SHAPE[1]))
nodes = [
helper.make_node('Constant', inputs=[], outputs=['v_in'], value=v_t),
helper.make_node('ArgMax', inputs=['v_in'], outputs=['v_out'],
axis=-1, keepdims=0),
]
out_vi = helper.make_tensor_value_info('v_out', TensorProto.INT64, [SHAPE[0]])
graph = helper.make_graph(nodes, 'minimal', [], [out_vi])
model = helper.make_model(graph, opset_imports=[helper.make_opsetid('', 11)])
# CUDA optimized (default)
sess_opt = ort.InferenceSession(
model.SerializeToString(),
providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],
)
# CUDA unoptimized (reference)
so_ref = ort.SessionOptions()
so_ref.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
sess_ref = ort.InferenceSession(
model.SerializeToString(),
providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],
sess_options=so_ref,
)
opt = np.asarray(sess_opt.run(None, {})[0])
ref = np.asarray(sess_ref.run(None, {})[0])
print(f"opt (optimizer on): {opt}") # [0 0 0 0]
print(f"ref (optimizer off): {ref}") # [2147483647 2147483647 2147483647 2147483647]
assert not np.allclose(opt.astype(np.float64), ref.astype(np.float64),
atol=0.5, rtol=0.1), "BUG DID NOT REPRODUCE"
```
### Urgency
No. I found this by fuzzing testing.
### Platform
Linux
### OS Version
Ubuntu 5.4.0-162-generic
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.27.0
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
CUDA
### Execution Provider Library Version
CUDA 13.0, Driver 580.76.05, GPU: NVIDIA RTX 3080 Ti
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the CUDAExecutionProvider ArgMax kernel entry point and compare its all--inf behavior with the optimizer's constant-folding result shown in the Python reproducer. Add or update coverage for a [4, 12] input filled with -inf, then run the CUDA ArgMax test path and verify the runtime returns the expected in-range result rather than INT32_MAX.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100