[Bug] maximum/minimum/relu/clip do not propagate NaN (IEEE 754 violation)
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 4k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 112
Description
## Expected behavior
`maximum(NaN, x)` should return `NaN` per IEEE 754-2019 §9.6, consistent with NumPy, PyTorch, JAX, and ONNX Runtime.
`relu(NaN)` should return `NaN` (since relu = max(x, 0)).
## Actual behavior
When NaN is the **first** operand of `T.max` / `T.min`, the result is the second operand instead of NaN. This affects `R.maximum`, `R.minimum`, `R.nn.relu`, and `R.clip`.
The root cause is that `T.max(a, b)` compiles to x86 `maxss`/`maxps` instructions, which have the hardware behavior: "if **src1** is NaN, return **src2**". IEEE 754 requires returning NaN when either operand is NaN.
## Reproducer
```python
import numpy as np
import tvm
from tvm import relax
import tvm.relax.op as R
from tvm.relax.transform import LegalizeOps
bb = relax.BlockBuilder()
a = relax.Var("a", relax.TensorStructInfo((4,), "float32"))
b = relax.Var("b", relax.TensorStructInfo((4,), "float32"))
with bb.function("main", [a, b]):
with bb.dataflow():
gv = bb.emit_output(bb.emit(R.maximum(a, b)))
bb.emit_func_output(gv)
mod = bb.finalize()
pipeline = tvm.ir.transform.Sequential([LegalizeOps()])
exe = tvm.relax.build(pipeline(mod), target="llvm")
vm = tvm.relax.VirtualMachine(exe, device=tvm.cpu())
A = np.array([np.nan, 1.0, np.nan, 0.0], np.float32)
B = np.array([1.0, np.nan, np.nan, np.nan], np.float32)
out = vm["main"](
tvm.runtime.tensor(A, device=tvm.cpu()),
tvm.runtime.tensor(B, device=tvm.cpu()),
).numpy()
print(out) # [1. nan nan nan] — element 0 is WRONG
print(np.maximum(A, B)) # [nan nan nan nan] — all NaN per IEEE 754
```
The pattern is operand-order-dependent:
| Expression | TVM | Expected (IEEE 754) |
|---|---|---|
| `max(NaN, 1.0)` | `1.0` | `NaN` |
| `max(1.0, NaN)` | `NaN` | `NaN` |
| `relu(NaN)` = `max(NaN, 0)` | `0.0` | `NaN` |
| `clip(NaN, -1, 1)` | `1.0` | `NaN` |
## Affected operations
```python
R.maximum(a, b) # when a is NaN
R.minimum(a, b) # when a is NaN
R.nn.relu(x) # when x is NaN → returns 0
R.clip(x, lo, hi) # when x is NaN → returns hi
```
Not affected (correct NaN propagation):
- `R.add`, `R.multiply`, `R.subtract`, `R.divide` — arithmetic propagates NaN correctly
- `R.nn.leakyrelu` — uses comparison path, NaN propagates through multiply
- `R.nn.silu`, `R.nn.gelu` — sigmoid/erf path propagates NaN
## Why this matters
`relu` is the most common activation function. When an upstream computation produces NaN (e.g., from overflow or division by zero), the NaN should propagate to signal the error. Instead, TVM's `relu` silently converts NaN to 0, making the error invisible:
```python
# Suppose upstream overflow produces NaN in one element:
x = [[1.0, 2.0, NaN, 4.0]]
relu(x).sum()
# TVM: 7.0 ← NaN silently disappeared
# NumPy: NaN ← correctly signals the problem
```
This can cause silent wrong results in production models, where NaN detection is a standard debugging/monitoring signal.
## Root cause
In the lowered TIR, `maximum` becomes `T.max(a, b)`, which LLVM lowers to x86 `maxss`/`maxps`. These instructions follow "if src1 is NaN, return src2" semantics rather than IEEE 754 "return NaN if either is NaN".
The fix would be to emit NaN-aware max/min, e.g.:
```
select(isnan(a) | isnan(b), NaN, max(a, b))
```
## Environment
- TVM commit: 0b0afd8dd (main, 2026-04-24)
- OS: Ubuntu 20.04
- Target: llvm (CPU, x86-64)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing how R.maximum, R.minimum, R.nn.relu, and R.clip are legalized to T.max/T.min and then lowered for the LLVM x86 target. Use the provided reproducer and operand table as regression coverage; done means NaN propagates for both operand orders and for relu and clip without changing the listed unaffected operations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100