microsoft / microsoft/onnxruntime

CUDA ArgMax on all -inf returns INT32_MAX sentinel value

Open
#31,161 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ep:CUDA stale
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

The CUDAExecutionProvider's ArgMax kernel returns `INT32_MAX` (2147483647) when ALL
elements along the reduction axis are `-inf`. The optimizer constant-folds this case
correctly (returns 0), but the runtime CUDA kernel produces an out-of-range sentinel.

### To reproduce

```python
from onnx import helper, TensorProto
import onnxruntime as ort
import numpy as np

SHAPE = [4, 12] # any [N, M] where M >= 2
v_t = helper.make_tensor("v_t", TensorProto.FLOAT, SHAPE,
[float('-inf')] * (SHAPE[0] * SHAPE[1]))

nodes = [
helper.make_node('Constant', inputs=[], outputs=['v_in'], value=v_t),
helper.make_node('ArgMax', inputs=['v_in'], outputs=['v_out'],
axis=-1, keepdims=0),
]

out_vi = helper.make_tensor_value_info('v_out', TensorProto.INT64, [SHAPE[0]])
graph = helper.make_graph(nodes, 'minimal', [], [out_vi])
model = helper.make_model(graph, opset_imports=[helper.make_opsetid('', 11)])

# CUDA optimized (default)
sess_opt = ort.InferenceSession(
model.SerializeToString(),
providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],
)

# CUDA unoptimized (reference)
so_ref = ort.SessionOptions()
so_ref.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
sess_ref = ort.InferenceSession(
model.SerializeToString(),
providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],
sess_options=so_ref,
)

opt = np.asarray(sess_opt.run(None, {})[0])
ref = np.asarray(sess_ref.run(None, {})[0])

print(f"opt (optimizer on): {opt}") # [0 0 0 0]
print(f"ref (optimizer off): {ref}") # [2147483647 2147483647 2147483647 2147483647]
assert not np.allclose(opt.astype(np.float64), ref.astype(np.float64),
atol=0.5, rtol=0.1), "BUG DID NOT REPRODUCE"
```

### Urgency

No. I found this by fuzzing testing.

### Platform

Linux

### OS Version

Ubuntu 5.4.0-162-generic

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.27.0

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

CUDA

### Execution Provider Library Version

CUDA 13.0, Driver 580.76.05, GPU: NVIDIA RTX 3080 Ti

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the CUDAExecutionProvider ArgMax kernel entry point and compare its all--inf behavior with the optimizer's constant-folding result shown in the Python reproducer. Add or update coverage for a [4, 12] input filled with -inf, then run the CUDA ArgMax test path and verify the runtime returns the expected in-range result rather than INT32_MAX.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.