microsoft / microsoft/onnxruntime

[Build] Encountered error executing OPs on RISCV platform

Open
#20,030 3 comments 0 reactions 0 assignees View on GitHub
contributions welcome
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

I am trying to build onnxruntime for a RISCV target platform ([LicheeRV Nano](https://wiki.sipeed.com/hardware/en/lichee/RV_Nano/1_intro.html)). I have succeeded the built of libonnxruntime.so using the cross compiler ([riscv64-unknown-linux-musl](https://sophon-file.sophon.cn/sophon-prod-s3/drive/23/03/07/16/host-tools.tar.gz)) and no other errors reported.
The same test project and model runs without errors on both Raspberry Pi (arm) and PC, the model accuracy is normal, but on the RISCV target platform it drops to 15% accuracy, which is equivalent to a completely wrong state.

I have tried several options, but they don't work:
-Donnxruntime_DISABLE_CONTRIB_OPS=ON
-Donnxruntime_DONT_VECTORIZE=ON
-Donnxruntime_DISABLE_ABSEIL=ON

The model is a simple 3-layer CNN including conv1d, relu and maxpool. I can at least confirm that the problem occurs after the first convolutional block (can't be sure it's the conv OP's fault).

I don't have any idea at the moment, so I'd appreciate if you could give me some ideas!

### Urgency

_No response_

### Target platform

RISCV (LicheeRV Nano)

### Build script

cmake ../cmake \
-Donnxruntime_GCC_STATIC_CPP_RUNTIME=ON \
-DCMAKE_BUILD_TYPE=Release \
-Donnxruntime_BUILD_SHARED_LIB=ON \
-Donnxruntime_BUILD_UNIT_TESTS=OFF \
-Donnxruntime_ENABLE_CPUINFO=OFF \
-DCMAKE_TOOLCHAIN_FILE=linux_lichee_crosscompile_toolchain.cmake

## linux_lichee_crosscompile_toolchain.cmake
SET(CMAKE_SYSTEM_NAME Linux)
SET(CMAKE_SYSTEM_VERSION 1)
set(tools "/opt/host-tools/gcc/riscv64-linux-musl-x86_64")
set(CMAKE_C_COMPILER ${tools}/bin/riscv64-unknown-linux-musl-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/riscv64-unknown-linux-musl-g++)
set(CMAKE_SYSTEM_PROCESSOR riscv64)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mcpu=c906fdv -march=rv64imafdcv0p7xthead -mcmodel=medany -mabi=lp64d")
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

### Error / output

RISCV platform:
ONNX Runtime version: 1.18.0
Input Node Name/Shape (0):
input : 1x9x128
Output Node Name/Shape (0):
output : -1x6
Accuracy: 15.23 %

PC platform:
ONNX Runtime version: 1.18.0
Input Node Name/Shape (0):
input : 1x9x128
Output Node Name/Shape (0):
output : -1x6
Accuracy: 86.09 %

### Visual Studio Version

_No response_

### GCC / Compiler Version

C++ compiler version : 10.2.0

Contributor guide

Open the contributing guide

Research direction

Start with the provided linux_lichee_crosscompile_toolchain.cmake and CMake build options, then compare the model output after the first convolutional block on RISCV and PC. Use the minimal CNN and its conv1d, relu, and maxpool path to isolate the failing operation; done means identifying a reproducible source of the accuracy divergence or narrowing it to a specific operator or compiler configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system, embedded-iot, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.