PaddlePaddle / PaddlePaddle/FastDeploy

option.set_trt_cache_file()API报错

Open
#2,076 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.7k
Forks
756
Avg merge
19h 28m
Merged PRs (30d)
4

Description

环境

  • 【FastDeploy版本】: jeston
  • 【编译命令】教程命令
  • 【系统平台】: Linux x64(Ubuntu 18.04) / Windows x64(Windows10) / Mac OSX arm(12.0) / Mac OSX intel(12.0)
  • 【硬件】: jeston nano
  • 【编译语言】: python3.6

问题日志及出现问题的操作流程

使用 option.set_trt_cache_file()API报错
错误信息 :

WARNING:root:`RuntimeOption.set_trt_input_shape` will be deprecated in v1.2.0, please use `RuntimeOption.trt_option.set_shape()` instead.
WARNING:root:`RuntimeOption.set_trt_cache_file` will be deprecated in v1.2.0, please use `RuntimeOption.trt_option.serialize_file = cache` instead.
[INFO] fastdeploy/vision/common/processors/transform.cc(45)::FuseNormalizeCast  Normalize and Cast are fused to Normalize in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW       Normalize and HWC2CHW are fused to NormalizeAndPermute  in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1
[INFO] fastdeploy/runtime/backends/tensorrt/trt_backend.cc(703)::CreateTrtEngineFromOnnx        Detect serialized TensorRT Engine file in cache, will load it directly.
Traceback (most recent call last):
  File "test.py", line 39, in <module>
    'img/ppyoloe_plus_crn_l_80e_coco/infer_cfg.yml',runtime_option=runtime_option)
  File "/home/paddle/.local/lib/python3.6/site-packages/fastdeploy/vision/detection/ppdet/__init__.py", line 115, in __init__
    model_format)
MemoryError: std::bad_alloc

具体代码:

import cv2
from fastdeploy.vision import detection
from fastdeploy import vision
import fastdeploy as fd

option = fd.RuntimeOption()

def build_option(device = 'cpu',backend = 'paddle'):
    
    # 创建Runtimeoption
    option = fd.RuntimeOption()
    option.use_cpu()
    if device.lower() == "gpu":
        option.use_gpu(0)

    if backend.lower() == "trt":
        assert device.lower(
        ) == "gpu", "TensorRT backend require inference on device GPU."
        option.use_trt_backend()
    elif backend.lower() == "pptrt":
        assert device.lower(
        ) == "gpu", "TensorRT backend require inference on device GPU."
        option.use_trt_backend()
        option.enable_paddle_to_trt()
    elif backend.lower() == "ort":
        option.use_ort_backend()
    elif backend.lower() == "paddle":
        option.use_paddle_backend()
    elif backend.lower() == "openvino":
        option.use_openvino_backend()
    option.set_trt_input_shape('input',[1,3,640,640])
    option.set_trt_cache_file('cache')
    return option
runtime_option = build_option(device = 'gpu',backend = 'trt')
#runtime_option.set_trt_input_shape('inpu',[1,3,640,640])
#runtime_option.set_trt_cache_file('model/ppyoloe_plus_crn_l_80e_coco')
model = detection.PPYOLOE('img/ppyoloe_plus_crn_l_80e_coco/model.pdmodel',
                          'img/ppyoloe_plus_crn_l_80e_coco/model.pdiparams',
                          'img/ppyoloe_plus_crn_l_80e_coco/infer_cfg.yml',runtime_option=runtime_option)
# model.preprocessor.use_cuda(True, 0)

cpa = cv2.VideoCapture(0)
ret, frame = cpa.read()  # 预先读取一帧用于初始化

while True:
    if ret:
        result = model.predict(frame)
        vis_im = vision.vis_detection(frame, result, score_threshold=0.699)
        cv2.imshow("vis_img.jpg", vis_im)
        print(result)
    
    key = cv2.waitKey(1)
    if key == 27:  # 按下 ESC 键退出循环
        break
    
    ret, frame = cpa.read()

cpa.release()
cv2.destroyAllWindows()```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with RuntimeOption.set_trt_cache_file and the PPYOLOE initialization shown in test.py, then reproduce the MemoryError using the supplied model paths and runtime settings. Trace the call into the TensorRT backend and compare the deprecated API warnings with the cache-loading failure; done means the example initializes successfully without std::bad_alloc or the supported limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.