apache / apache/tvm

[Bug] TVM produces wrong result with the "default" optimization pipeline

Open
#17,835 0 comments 0 reactions 0 assignees View on GitHub
needs-triage type: bug
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 19h
Merged PRs (30d)
111

Description

### Expected behavior
For the following onnx model, the output of "v6_0" should be 0.
```python
[array(518, dtype=int64),
array([ True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True]),
array(0, dtype=int64),
array([25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25], dtype=uint8)]
```

![Image](https://github.com/user-attachments/assets/0d96dbae-8aa1-49f6-9c1e-756825b93a7a)

### Actual behavior

However, when we compile the onnx model using relax.build with the default optimization pipeline, tvm produces -1 of v6_0.
```python
(
array(518),
array([ True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True]),

array(-1),

array([25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25], dtype=uint8))
```
### Environment

OS: Ubuntu 20.04
TVM: 0.20.dev0 (f6236ce41)

### Steps to reproduce

This bug can be reproduced by the following code with the model in the attachment.
```python
from typing import Dict, List, Literal, Optional

import numpy as np
import onnx
import onnxruntime
from onnx import ModelProto, TensorProto, helper, mapping

import tvm
import tvm.testing
from tvm import relax
from tvm.relax.frontend.onnx import from_onnx

import pickle

def get_oracle(oracle_path):
with open(oracle_path, 'rb') as f:
oracle = pickle.load(f)

return oracle['input'], oracle['output']

def check(
model: ModelProto,
inputs: Optional[Dict[str, np.ndarray]] = None,
) -> None:

# Run the model through onnx to get the expected result.
try:
ort_session = onnxruntime.InferenceSession(
model.SerializeToString(), providers=["CPUExecutionProvider"]
)
ort_output = ort_session.run([], inputs)
except:
print("This model cannot be executed by onnxruntime!")
sys.exit(1)

print("onnxrumtime: ", ort_output)
# Convert the onnx model into relax through the onnx importer.
tvm_model = from_onnx(model, keep_params_in_input=True)
# Convert operators for inference mode.
tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model)
# Legalize any relax ops into tensorir.
tvm_model = relax.transform.LegalizeOps()(tvm_model)

# Separate model from parameters.
tvm_model, params = relax.frontend.detach_params(tvm_model)

# Prepare inputs.
input_list = [
inputs[key.name_hint] for key in tvm_model["main"].params if key.name_hint in inputs
]
if params:
input_list += params["main"]

# Compile the relax graph into a VM then run.
#----------------------cpu-----------------------
with tvm.transform.PassContext(opt_level=0):

ex = relax.build(tvm_model, target="llvm", relax_pipeline="default")
vm = relax.VirtualMachine(ex, tvm.cpu())

# Run model and check outputs.
vm.set_input("main", *input_list)
vm.invoke_stateful("main")
tvm_cpu_output = vm.get_outputs("main")
#----------------------cpu-----------------------
print("tvm cpu", tvm_cpu_output)


def main(model_path, oracle_path):
onnx_model = onnx.load(model_path)
inputs, outputs = get_oracle(oracle_path)

check(onnx_model, inputs=inputs)

main("model.onnx", "oracle.pkl")

```

[testcase.zip](https://github.com/user-attachments/files/19732016/testcase.zip)

### Triage

Please refer to the list of label tags [here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the relevant tags and add them below in a bullet format (example below).

* needs-triage

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the Python reproducer with testcase.zip and compare the ONNX Runtime output with the result from relax.build using the default optimization pipeline. Trace the Relax ONNX import, DecomposeOpsForInference, LegalizeOps, and default pipeline stages to identify where v6_0 changes from 0 to -1. Done means the reproduced model returns 0 for v6_0 while preserving the other outputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.