apache / apache/tvm

[Bug][LLVM] Relax constant parameters produce incorrect LLVM results on Windows

Open
#20,007 0 comments 0 reactions 0 assignees View on GitHub
needs-triage type: bug
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 1h
Merged PRs (30d)
112

Description

### Expected behavior

Embedding `float32` parameter values as Relax constants, or binding equivalent values with `Function.bind_params`, should preserve the numerical result of a Relax function compiled for LLVM.

Passing the same values as explicit VM arguments should be numerically equivalent to the embedded-constant form.

### Actual behavior

On Windows, a pure Relax program produces an incorrect result when its weight and twelve broadcast bias vectors are embedded as `relax.const` values.

The same program is correct when those values are passed as explicit VM arguments.

For the minimal reproducer below, the embedded-constant path produces `-2.010380268096924`, while the NumPy reference is `-2.289207935333252`.

The explicit-parameter path has a maximum absolute difference of `0` for the same input values.

The minimal reproducer does not involve a model frontend.

### Environment

- OS: Windows 11
- Python: 3.11.14
- TVM: 0.25.0
- Build: local Release build with LLVM `22.1.8`, target `llvm`
- Device: CPU

### Steps to reproduce

Save the following as `repro_relax_const.py` and run it with the Python environment that imports the matching TVM source and compiled libraries.

```python
import numpy as np

import tvm
from tvm import relax
from tvm.runtime import tensor as tvm_tensor

def run(embed_as_constants):
rng = np.random.default_rng(0)
x_value = rng.normal(size=(1, 1)).astype("float32")
weight_value = rng.normal(size=(1, 1)).astype("float32")
bias_values = [rng.normal(size=(1,)).astype("float32") for _ in range(12)]

expected = x_value @ weight_value
for bias_value in bias_values:
expected = expected + bias_value

sinfo_x = relax.TensorStructInfo((1, 1), "float32")
sinfo_w = relax.TensorStructInfo((1, 1), "float32")
sinfo_b = relax.TensorStructInfo((1,), "float32")
x = relax.Var("x", sinfo_x)
weight = relax.Var("weight", sinfo_w)
biases = [relax.Var(f"bias_{i}", sinfo_b) for i in range(12)]

if embed_as_constants:
weight_expr = relax.const(weight_value)
bias_exprs = [relax.const(value) for value in bias_values]
function_params = [x]
else:
weight_expr = weight
bias_exprs = biases
function_params = [x, weight, *biases]

bb = relax.BlockBuilder()
with bb.function("main", function_params):
with bb.dataflow():
result = bb.emit(relax.op.matmul(x, weight_expr))
for bias_expr in bias_exprs:
result = bb.emit(relax.op.add(result, bias_expr))
output = bb.emit_output(result)
bb.emit_func_output(output)
mod = bb.finalize()

executable = relax.build(mod, target="llvm")
vm = relax.VirtualMachine(executable, tvm.cpu(0))
arguments = [tvm_tensor(x_value, tvm.cpu(0))]
if not embed_as_constants:
arguments.extend([tvm_tensor(weight_value, tvm.cpu(0))])
arguments.extend(tvm_tensor(value, tvm.cpu(0)) for value in bias_values)
actual = vm["main"](*arguments).numpy()
print(f"embed_as_constants={embed_as_constants}")
print("expected:", expected)
print("actual: ", actual)
np.testing.assert_allclose(actual, expected, rtol=1e-5, atol=1e-5)

run(False) # Passes: maximum absolute difference is 0.
run(True) # Fails: maximum absolute difference is about 0.27882767.
```

`run(False)` passes.

`run(True)` fails with:

```text
ACTUAL: [[-2.0103803]]
DESIRED: [[-2.289208]]
Max absolute difference: 0.27882767
```

I also reproduced the same parameter-mode distinction with an ONNX DenseNet-121 model on the same Windows environment.

`from_onnx(..., keep_params_in_input=False)` produced a maximum absolute difference of `0.848955` against ONNX Runtime, while `keep_params_in_input=True` passed with a maximum absolute difference of `3.58e-06`.

This ONNX result is supporting evidence only; the pure Relax script above is the intended minimal reproducer.

### Triage

* needs-triage
* type: bug

Contributor guide

No contributing guide indexed for this repository

Research direction

Save the minimal reproducer as repro_relax_const.py and run it in the matching TVM environment on Windows. Compare the embedded-constant and explicit-parameter paths, then verify that the embedded path matches the NumPy reference and that the provided np.testing.assert_allclose check passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.