apache / apache/tvm

[Bug] Matmul and reductions with zero-size dimension return uninitialized memory

Open
#19,578 0 comments 0 reactions 0 assignees View on GitHub
needs-triage type: bug
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 1h
Merged PRs (30d)
112

Description

## Expected behavior

`matmul(A[M,0], B[0,N])` should return an M×N matrix of zeros (sum of zero terms = 0), consistent with NumPy, PyTorch, and ONNX Runtime.

## Actual behavior

The output buffer is not zeroed, so the result contains uninitialized memory (garbage values, NaN). The output is non-deterministic across runs.

## Reproducer

```python
import numpy as np
import tvm
from tvm import relax
import tvm.relax.op as R
from tvm.relax.transform import LegalizeOps

bb = relax.BlockBuilder()
a = relax.Var("a", relax.TensorStructInfo((4, 0), "float32"))
b = relax.Var("b", relax.TensorStructInfo((0, 4), "float32"))
with bb.function("main", [a, b]):
with bb.dataflow():
gv = bb.emit_output(bb.emit(R.matmul(a, b)))
bb.emit_func_output(gv)
mod = bb.finalize()

pipeline = tvm.ir.transform.Sequential([LegalizeOps()])
exe = tvm.relax.build(pipeline(mod), target="llvm")
vm = tvm.relax.VirtualMachine(exe, device=tvm.cpu())

out = vm["main"](
tvm.runtime.tensor(np.zeros((4, 0), np.float32), device=tvm.cpu()),
tvm.runtime.tensor(np.zeros((0, 4), np.float32), device=tvm.cpu()),
).numpy()

print(out)
# Non-deterministic garbage, e.g.:
# [[ 6.19e+21 3.07e-41 5.25e+21 3.07e-41]
# [ 6.19e+21 3.07e-41 5.25e+21 3.07e-41]
# [ nan -7.43e-44 nan -1.40e-44]
# [ nan 7.57e-44 nan 4.48e-44]]

print(np.zeros((4, 0)) @ np.zeros((0, 4)))
# [[0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]
```

## Scope

The same issue affects other reduction operations with a zero-size reduction axis:

```python
# All return uninitialized memory instead of the identity element:
R.sum(X[4, 0], axis=1) # expected: zeros(4,1)
R.prod(X[4, 0], axis=1) # expected: ones(4,1)
R.matmul(A[2,3,4,0], B[2,3,0,5]) # expected: zeros(2,3,4,5) [batched]
R.nn.conv2d(X[1,0,8,8], W[4,0,3,3]) # expected: zeros(1,4,8,8) [0 in_channels]
R.einsum([A[4,0], B[0,8]], 'ik,kj->ij') # expected: zeros(4,8)
```

Notably, `R.max` and `R.min` over size-0 axes are handled correctly (returning -inf / +inf).

## Why this matters

The garbage output propagates silently. For example:

```python
mm = R.matmul(A[4,0], B[0,8]) # garbage
sm = R.nn.softmax(mm, axis=-1) # produces [1,0,0,...,0] instead of uniform
```

In dynamic-shape models, K=0 can occur naturally when a filter removes all elements. The result is silent, non-deterministic incorrect output with no crash or warning.

## Environment

- TVM commit: 0b0afd8dd (main, 2026-04-24)
- OS: Ubuntu 20.04
- Target: llvm (CPU)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Relax operations exercised by LegalizeOps and the LLVM target in the supplied reproducer. Trace matmul and reduction operations with zero-size reduction axes, then verify that the listed cases return their identity values, including batched matmul, convolution, and einsum.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.