PaddlePaddle / PaddlePaddle/FastDeploy

Custom trained yolov5/8 on rknpu2 for RK3588

Open
#1,888 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.7k
Forks
756
Avg merge
19h 28m
Merged PRs (30d)
4

Description

Environment

FastDeploy version: 1.0.6
OS Platform: e.g. ubuntu Linux x64 & debian 11 on rk3588s board
Hardware: e.g. Nvidia GPU 3070Ti CUDA 11.2 CUDNN 8.3
Program Language: e.g. Python 3.8

Problem description

I am trying to train my custom yolov5/v8 model, convert it to rknn format and then run inference on the same.
Can someone explain the steps for me to find out what I am doing wrong.

I am trying to convert the default models from onnx to rknn using the fastdeploy tools/rknpu2 step. It fails with the following error
"""
{'mean': [[0, 0, 0]], 'std': [[255, 255, 255]], 'model_path': './yolov8n.onnx', 'outputs_nodes': ['p2o.Mul.1', 'p2o.Concat.49'], 'do_quantization': False, 'dataset': './coco_dataset_20.txt', 'output_folder': './yolov8_n_500e_coco'}
W init: rknn-toolkit2 version: 1.4.0-22dcfef4
W load_onnx: If you don't need to crop the model, don't set 'inputs'/'input_size_list'/'outputs'!
E load_onnx: The 'p2o.Mul.1' in outputs=['p2o.Mul.1', 'p2o.Concat.49'] is invalid!
W load_onnx: ===================== WARN(2) =====================
E rknn-toolkit2 version: 1.4.0-22dcfef4
E load_onnx: Catch exception when loading onnx model: /home/dt/Projects/Work/greenox/rknn/yolov8onnxtorknn/FastDeploy/tools/rknpu2/yolov8n.onnx!
E load_onnx: Traceback (most recent call last):
E load_onnx: File "rknn/api/rknn_base.py", line 1136, in rknn.api.rknn_base.RKNNBase.load_onnx
E load_onnx: File "rknn/api/rknn_log.py", line 113, in rknn.api.rknn_log.RKNNLog.e
E load_onnx: ValueError: The 'p2o.Mul.1' in outputs=['p2o.Mul.1', 'p2o.Concat.49'] is invalid!
Traceback (most recent call last):
File "export.py", line 52, in
assert ret == 0, "Load model failed!"
AssertionError: Load model failed!"""

I tried using the rknn_model_zoo convert functionality, but it also only converts the standard models given by them.

Does your rkyolo demo at (https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/vision/detection/rkyolo) work on RK3588 and on custom models? If so how can i train custom and run inference on RK3588.

Thank you for your help in advance!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing FastDeploy/tools/rknpu2/export.py and the referenced rknn_model_zoo conversion flow, then compare them with the rkyolo demo. Reproduce the reported invalid output-node failure using yolov8n.onnx and inspect how custom YOLO models are converted. Done means documenting or validating a working custom-model conversion and RK3588 inference path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai-infra-agents, embedded-iot, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.