[Bug] TVM MIPS32 .so Model Load Failure – ONNX Model Converted to MIPS32 Fails to Load with TVM Runtime
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 4k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 111
Description
### Expected behavior
I expect `model.so` (a TVM-compiled model for MIPS32) to be successfully loaded using `dlopen` or `TVMModLoadFromFile` on my MIPS32 development board, given that `libtvm_runtime.so` loads and runs correctly
### Actual behavior
- `libtvm_runtime.so` (v0.18.0) loads successfully on the MIPS32 board and executes basic runtime tests.
- However, `model.so` fails to load with `dlopen`:
```
Failed to load: /path/to/model.so: cannot open shared object file: No such file or directory
```
- Same error occurs with `TVMModLoadFromFile`, even though the file exists at that path and has correct permissions.
### Environment
**Operating System:**
- Host: Ubuntu 20.04, x86_64
- Target: MIPS32-based development board
**TVM Version:** v0.18.0
**Compiler:**
- Host: GCC (x86_64)
- Target: MIPS cross-compiler (`mips-linux-gnu-gcc`, GCC 7.2.0, glibc 2.29)
**TVM Build Configuration:**
### x86_64 (Host)
```bash
#build on x86 PC
cd /path/to/tvm
git checkout v0.18.0
mkdir build-x86_64 && cd build-x86_64
cmake -DCMAKE_BUILD_TYPE=Release -DUSE_LLVM=ON ..
make -j4
```
- Output: `libtvm.so`
### MIPS32 (Target)
**`mips32el-toolchain.cmake.cmake`**
```cmake
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR mipsel)
set(CMAKE_C_COMPILER /opt/mips-gcc720-glibc229/bin/mips-linux-gnu-gcc)
set(CMAKE_CXX_COMPILER /opt/mips-gcc720-glibc229/bin/mips-linux-gnu-g++)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
```
```bash
#build on x86 PC
cd /path/to/tvm
git checkout v0.18.0
mkdir build-mips32 && cd build-mips32
cmake \
-DCMAKE_TOOLCHAIN_FILE=../mips32el-toolchain.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DUSE_LLVM=OFF \
-DUSE_RPC=OFF \
-DUSE_GRAPH_EXECUTOR=OFF \
-DUSE_PROFILER=OFF \
-DUSE_LIBBACKTRACE=OFF \
..
make -j4
```
- Output: `libtvm_runtime.so`
---
## Cross-Compilation for Model
**Python script (`generate_onnx.py`)**
```python
import torch
import torch.nn as nn
import torch.onnx
class SimpleCNN(nn.Module):
def __init__(self):
super(SimpleCNN, self).__init__()
self.conv1 = nn.Conv2d(1, 16, 3)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(16, 32, 3)
self.fc1 = nn.Linear(32 * 5 * 5, 10)
def forward(self, x):
x = self.pool(torch.relu(self.conv1(x)))
x = self.pool(torch.relu(self.conv2(x)))
x = x.view(-1, 32 * 5 * 5)
x = self.fc1(x)
return x
model = SimpleCNN()
model.eval()
dummy_input = torch.randn(1, 1, 28, 28)
torch.onnx.export(model, dummy_input, "model.onnx",
input_names=["input"], output_names=["output"],
opset_version=11)
print("ONNX model exported as 'model.onnx'")
```
**Python script (`generate_so.py`)**
```python
import tvm
from tvm import relay
import onnx
from tvm.contrib import graph_executor
import tvm.contrib.cc as cc
onnx_model = onnx.load("model.onnx")
input_shape = (1, 1, 28, 28)
mod, params = relay.frontend.from_onnx(onnx_model, {"input": input_shape})
target = "llvm -mtriple=mipsel-linux-gnu -mcpu=mips32r2"
with tvm.transform.PassContext(opt_level=3):
lib = relay.build(mod, target=target, params=params)
cross_compiler = cc.cross_compiler(
"/path/to/mips-gcc720-glibc229/bin/mips-linux-gnu-gcc",
options=["-mfp32", "-mnan=legacy"]
)
lib.export_library("model.so", fcompile=cross_compiler)
```
- Flags: `-mfp32 -mnan=legacy`
---
### Steps to reproduce
### C Test: `test_dlopen.c`
```c
#include
#include
int main() {
void* handle = dlopen("/path/to/model.so", RTLD_LAZY);
if (!handle) {
printf("Failed to load: %s\n", dlerror());
return -1;
}
printf("Loaded successfully\n");
dlclose(handle);
return 0;
}
```
- Compile:
```bash
/path/to/mips-gcc720-glibc229/bin/mips-linux-gnu-gcc -o test_dlopen test_dlopen.c -ldl
```
---
### C++ Test: `test_tvm_runtime.cpp`
```cpp
#include
#include
int main() {
std::cout << "TVM Runtime Version: " << TVM_VERSION << std::endl;
TVMAPISetLastError("Test error message");
const char* error = TVMGetLastError();
std::cout << "Last Error: " << (error ? error : "None") << std::endl;
std::cout << "Test completed successfully!" << std::endl;
return 0;
}
```
- Compile:
```bash
/path/to/mips-gcc720-glibc229/bin/mips-linux-gnu-g++ -o test_tvm_runtime test_tvm_runtime.cpp \
-I /path/to/tvm/include \
-I /path/to/tvm/3rdparty/dlpack/include \
-I /path/to/tvm/3rdparty/dmlc-core/include \
-L /path/to/tvm/build-mips32 \
-ltvm_runtime -pthread -std=c++17
```
---
## Deploy and Run on MIPS32
```bash
cd /path/to/XXX
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/XXX
chmod +x test_dlopen test_tvm_runtime
./test_tvm_runtime # Works fine
./test_dlopen # Fails
```
---
## Output
```
test_tvm_runtime:
TVM Runtime Version: 0.18.0
Last Error: Test error message
Test completed successfully!
test_dlopen:
Failed to load: /path/to/XXX/model.so: cannot open shared object file: No such file or directory
```
## More Info
```bash
file model.so
model.so: ELF 32-bit LSB shared object, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, with debug_info, not stripped
file libtvm_runtime.so
libtvm_runtime.so: ELF 32-bit LSB shared object, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, with debug_info, not stripped
file libtvm.so
libtvm.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=ac5555851eefd24f951e85366f8fd0a5c95987d3, not stripped
```
```
If running on a PC x86 platform, the model.so can be successfully loaded and executed.
```
### Triage
*compilation*
*target:mips*
*component:runtime*
*area:relay*
*bug*
Contributor guide
No contributing guide indexed for this repository
Research direction
Begin with generate_so.py and mips32el-toolchain.cmake, then compare the ELF metadata and loader error for model.so and libtvm_runtime.so. Reproduce with test_dlopen.c and test_tvm_runtime.cpp on the MIPS32 board. Done means identifying the loading incompatibility and providing a model artifact or build configuration that loads successfully with the TVM runtime.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, cpp, python, pytorch
- Domain
- compilers, machine-learning, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100