[Bug][LLVM] Relax constant parameters produce incorrect LLVM results on Windows
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 4k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 112
Description
### Expected behavior
Embedding `float32` parameter values as Relax constants, or binding equivalent values with `Function.bind_params`, should preserve the numerical result of a Relax function compiled for LLVM.
Passing the same values as explicit VM arguments should be numerically equivalent to the embedded-constant form.
### Actual behavior
On Windows, a pure Relax program produces an incorrect result when its weight and twelve broadcast bias vectors are embedded as `relax.const` values.
The same program is correct when those values are passed as explicit VM arguments.
For the minimal reproducer below, the embedded-constant path produces `-2.010380268096924`, while the NumPy reference is `-2.289207935333252`.
The explicit-parameter path has a maximum absolute difference of `0` for the same input values.
The minimal reproducer does not involve a model frontend.
### Environment
- OS: Windows 11
- Python: 3.11.14
- TVM: 0.25.0
- Build: local Release build with LLVM `22.1.8`, target `llvm`
- Device: CPU
### Steps to reproduce
Save the following as `repro_relax_const.py` and run it with the Python environment that imports the matching TVM source and compiled libraries.
```python
import numpy as np
import tvm
from tvm import relax
from tvm.runtime import tensor as tvm_tensor
def run(embed_as_constants):
rng = np.random.default_rng(0)
x_value = rng.normal(size=(1, 1)).astype("float32")
weight_value = rng.normal(size=(1, 1)).astype("float32")
bias_values = [rng.normal(size=(1,)).astype("float32") for _ in range(12)]
expected = x_value @ weight_value
for bias_value in bias_values:
expected = expected + bias_value
sinfo_x = relax.TensorStructInfo((1, 1), "float32")
sinfo_w = relax.TensorStructInfo((1, 1), "float32")
sinfo_b = relax.TensorStructInfo((1,), "float32")
x = relax.Var("x", sinfo_x)
weight = relax.Var("weight", sinfo_w)
biases = [relax.Var(f"bias_{i}", sinfo_b) for i in range(12)]
if embed_as_constants:
weight_expr = relax.const(weight_value)
bias_exprs = [relax.const(value) for value in bias_values]
function_params = [x]
else:
weight_expr = weight
bias_exprs = biases
function_params = [x, weight, *biases]
bb = relax.BlockBuilder()
with bb.function("main", function_params):
with bb.dataflow():
result = bb.emit(relax.op.matmul(x, weight_expr))
for bias_expr in bias_exprs:
result = bb.emit(relax.op.add(result, bias_expr))
output = bb.emit_output(result)
bb.emit_func_output(output)
mod = bb.finalize()
executable = relax.build(mod, target="llvm")
vm = relax.VirtualMachine(executable, tvm.cpu(0))
arguments = [tvm_tensor(x_value, tvm.cpu(0))]
if not embed_as_constants:
arguments.extend([tvm_tensor(weight_value, tvm.cpu(0))])
arguments.extend(tvm_tensor(value, tvm.cpu(0)) for value in bias_values)
actual = vm["main"](*arguments).numpy()
print(f"embed_as_constants={embed_as_constants}")
print("expected:", expected)
print("actual: ", actual)
np.testing.assert_allclose(actual, expected, rtol=1e-5, atol=1e-5)
run(False) # Passes: maximum absolute difference is 0.
run(True) # Fails: maximum absolute difference is about 0.27882767.
```
`run(False)` passes.
`run(True)` fails with:
```text
ACTUAL: [[-2.0103803]]
DESIRED: [[-2.289208]]
Max absolute difference: 0.27882767
```
I also reproduced the same parameter-mode distinction with an ONNX DenseNet-121 model on the same Windows environment.
`from_onnx(..., keep_params_in_input=False)` produced a maximum absolute difference of `0.848955` against ONNX Runtime, while `keep_params_in_input=True` passed with a maximum absolute difference of `3.58e-06`.
This ONNX result is supporting evidence only; the pure Relax script above is the intended minimal reproducer.
### Triage
* needs-triage
* type: bug
Contributor guide
No contributing guide indexed for this repository
Research direction
Save the minimal reproducer as repro_relax_const.py and run it in the matching TVM environment on Windows. Compare the embedded-constant and explicit-parameter paths, then verify that the embedded path matches the NumPy reference and that the provided np.testing.assert_allclose check passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100