microsoft / microsoft/onnxruntime
[Build] Encountered error executing OPs on RISCV platform
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
I am trying to build onnxruntime for a RISCV target platform ([LicheeRV Nano](https://wiki.sipeed.com/hardware/en/lichee/RV_Nano/1_intro.html)). I have succeeded the built of libonnxruntime.so using the cross compiler ([riscv64-unknown-linux-musl](https://sophon-file.sophon.cn/sophon-prod-s3/drive/23/03/07/16/host-tools.tar.gz)) and no other errors reported.
The same test project and model runs without errors on both Raspberry Pi (arm) and PC, the model accuracy is normal, but on the RISCV target platform it drops to 15% accuracy, which is equivalent to a completely wrong state.
I have tried several options, but they don't work:
-Donnxruntime_DISABLE_CONTRIB_OPS=ON
-Donnxruntime_DONT_VECTORIZE=ON
-Donnxruntime_DISABLE_ABSEIL=ON
The model is a simple 3-layer CNN including conv1d, relu and maxpool. I can at least confirm that the problem occurs after the first convolutional block (can't be sure it's the conv OP's fault).
I don't have any idea at the moment, so I'd appreciate if you could give me some ideas!
### Urgency
_No response_
### Target platform
RISCV (LicheeRV Nano)
### Build script
cmake ../cmake \
-Donnxruntime_GCC_STATIC_CPP_RUNTIME=ON \
-DCMAKE_BUILD_TYPE=Release \
-Donnxruntime_BUILD_SHARED_LIB=ON \
-Donnxruntime_BUILD_UNIT_TESTS=OFF \
-Donnxruntime_ENABLE_CPUINFO=OFF \
-DCMAKE_TOOLCHAIN_FILE=linux_lichee_crosscompile_toolchain.cmake
## linux_lichee_crosscompile_toolchain.cmake
SET(CMAKE_SYSTEM_NAME Linux)
SET(CMAKE_SYSTEM_VERSION 1)
set(tools "/opt/host-tools/gcc/riscv64-linux-musl-x86_64")
set(CMAKE_C_COMPILER ${tools}/bin/riscv64-unknown-linux-musl-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/riscv64-unknown-linux-musl-g++)
set(CMAKE_SYSTEM_PROCESSOR riscv64)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mcpu=c906fdv -march=rv64imafdcv0p7xthead -mcmodel=medany -mabi=lp64d")
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
### Error / output
RISCV platform:
ONNX Runtime version: 1.18.0
Input Node Name/Shape (0):
input : 1x9x128
Output Node Name/Shape (0):
output : -1x6
Accuracy: 15.23 %
PC platform:
ONNX Runtime version: 1.18.0
Input Node Name/Shape (0):
input : 1x9x128
Output Node Name/Shape (0):
output : -1x6
Accuracy: 86.09 %
### Visual Studio Version
_No response_
### GCC / Compiler Version
C++ compiler version : 10.2.0
Contributor guide
Research direction
Start with the provided linux_lichee_crosscompile_toolchain.cmake and CMake build options, then compare the model output after the first convolutional block on RISCV and PC. Use the minimal CNN and its conv1d, relu, and maxpool path to isolate the failing operation; done means identifying a reproducible source of the accuracy divergence or narrowing it to a specific operator or compiler configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system, embedded-iot, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100