apache / apache/tvm

[Bug] InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure

Open
#18,132 3 comments 0 reactions 0 assignees View on GitHub
needs-triage type: bug
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 1h
Merged PRs (30d)
112

Description

### Expected behavior

TVM should run the model correctly.

### Actual behavior
For the following model,

Image

it can be executed by onnxruntime, the results are as follows:
```c
ONNXRuntime:
[array([[[[False, False, False, False],
[False, False, False, False],
[False, False, False, False],
[False, False, False, False]]]])]
```
However, when compiling and running the model using TVM, TVM crashes:
```c
File "/home/carla/Documents/tvm/python/tvm/runtime/vm.py", line 295, in invoke_stateful
self._invoke_stateful(func_name)
File "tvm/ffi/cython/./function.pxi", line 228, in tvm.ffi.core.Function.__call__
tvm.error.InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure, attempted to allocate 18446744073709551553 at offset 0 in region that is 0bytes
```

### Environment

OS: Ubuntu 20.04
TVM: 0.22.dev0 (c6969d723)
onnxruntime: 1.21.0

### Steps to reproduce

This bug can be reproduced by the following code with the model in the attachment. As shown in the code, the model can be executed by onnxruntime. However, TVM crashes when calling the invoke_stateful function.
```python
import sys

import numpy as np
import onnx
import onnxruntime

import tvm
from tvm import relax
from tvm.relax.frontend.onnx import from_onnx

import pickle


def main():
onnx_model = onnx.load("111.onnx")

with open("inputs.pkl", "rb") as fp:
inputs = pickle.load(fp)
print(inputs)
try:
ort_session = onnxruntime.InferenceSession(
onnx_model.SerializeToString(), providers=["CPUExecutionProvider"]
)
ort_output = ort_session.run([], inputs)
except Exception as e:
print(e)
sys.exit(1)

print("ONNXRuntime:\n", ort_output)

# Convert the onnx model into relax through the onnx importer.
tvm_model = from_onnx(onnx_model, keep_params_in_input=True)
# Convert operators for inference mode.
tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model)
# Legalize any relax ops into tensorir.
tvm_model = relax.transform.LegalizeOps()(tvm_model)

# Separate model from parameters.
tvm_model, params = relax.frontend.detach_params(tvm_model)

# Prepare inputs.
input_list = [
inputs[key.name_hint] for key in tvm_model["main"].params if key.name_hint in inputs
]
if params:
input_list += params["main"]

# Compile the relax graph into a VM then run.
#----------------------cpu-----------------------
with tvm.transform.PassContext(opt_level=0):
target = tvm.target.Target("llvm", host="llvm")
relax_pipeline = relax.pipeline.get_default_pipeline(target)

ex = relax.build(tvm_model, target="llvm", relax_pipeline=relax_pipeline)
vm = relax.VirtualMachine(ex, tvm.cpu())

# Run model and check outputs.
vm.set_input("main", *input_list)
vm.invoke_stateful("main")
tvm_cpu_output = vm.get_outputs("main")


if __name__ == "__main__":

main()

```

[testcase.zip](https://github.com/user-attachments/files/21178457/testcase.zip)

### Triage

Please refer to the list of label tags [here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the relevant tags and add them below in a bullet format (example below).

* needs-triage

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the supplied testcase.zip and run the Python reproduction, then inspect python/tvm/runtime/vm.py around invoke_stateful and the relax.frontend.onnx.from_onnx and relax.build entry points. Done means the attached model runs through vm.invoke_stateful without the storage allocation failure and produces the expected output shown from ONNXRuntime.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.