PaddlePaddle / PaddlePaddle/FastDeploy
fastdeploy 部署paddleocr,压力测试异常,进程退出
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.7k
- Forks
- 756
- Avg merge
- 19h 28m
- Merged PRs (30d)
- 4
Description
温馨提示:根据社区不完全统计,按照模板提问,可以加快回复和解决问题的速度
环境
【FastDeploy版本】: docker镜像 registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
【系统平台】: Linux x64(Ubuntu 22.04)
【硬件】: Nvidia GPU P40, CUDA 11.7 CUDNN 8.4
【编译语言】:Python3.8
问题日志及出现问题的操作流程
- 附上详细的问题日志有助于快速定位分析
call ocr
0# 0x00005650067998A9 in fastdeployserver
1# 0x00007FAF577AF210 in /usr/lib/x86_64-linux-gnu/libc.so.6
2# void phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<int, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >) in /opt/fastdeploy/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so
3# void phi::KernelImpl<void ()(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<int, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >), &(void phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<int, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >))>::KernelCallHelper<paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<int, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >, phi::TypeTag >::Compute<1, 3, 0, 0, phi::GPUContext const, phi::DenseTensor const, phi::DenseTensor const, phi::DenseTensor const>(phi::KernelContext*, phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&) in /opt/fastdeploy/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so
4# paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, phi::Place const&, paddle::framework::RuntimeContext*) const in /opt/fastdeploy/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so
5# paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, phi::Place const&) const in /opt/fastdeploy/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so
6# paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, phi::Place const&) in /opt/fastdeploy/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so
7# 0x00007FAE3D72085D in /opt/fastdeploy/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so
8# paddle::AnalysisPredictor::ZeroCopyRun() in /opt/fastdeploy/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so
9# fastdeploy::PaddleBackend::Infer(std::vector<fastdeploy::FDTensor, std::allocatorfastdeploy::FDTensor >&, std::vector<fastdeploy::FDTensor, std::allocatorfastdeploy::FDTensor >*, bool) in /opt/fastdeploy/lib/libfastdeploy_runtime.so.0.0.0
10# fastdeploy::Runtime::Infer() in /opt/fastdeploy/lib/libfastdeploy_runtime.so.0.0.0
11# 0x00007FAEFC1B0FA4 in /opt/tritonserver/backends/fastdeploy/libtriton_fastdeploy.so
12# 0x00007FAEFC1B4826 in /opt/tritonserver/backends/fastdeploy/libtriton_fastdeploy.so
13# TRITONBACKEND_ModelInstanceExecute in /opt/tritonserver/backends/fastdeploy/libtriton_fastdeploy.so
14# 0x00007FAF5833B83A in /opt/tritonserver/bin/../lib/libtritonserver.so
15# 0x00007FAF5833C04D in /opt/tritonserver/bin/../lib/libtritonserver.so
16# 0x00007FAF581F0801 in /opt/tritonserver/bin/../lib/libtritonserver.so
17# 0x00007FAF58335DC7 in /opt/tritonserver/bin/../lib/libtritonserver.so
18# 0x00007FAF57B9DDE4 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6
19# 0x00007FAF5801B609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0
20# clone in /usr/lib/x86_64-linux-gnu/libc.so.6
[StatusCode.UNAVAILABLE] recvmsg:Connection reset by peer
[StatusCode.UNAVAILABLE] recvmsg:Connection reset by peer
[StatusCode.UNAVAILABLE] recvmsg:Connection reset by peer
[StatusCode.UNAVAILABLE] recvmsg:Connection reset by peer
[StatusCode.UNAVAILABLE] recvmsg:Connection reset by peer
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the OCR call under the reported pressure test in the FastDeploy Triton backend, then inspect the stack from TRITONBACKEND_ModelInstanceExecute through PaddleBackend::Infer and Runtime::Infer. Done means the process remains available during the test and the reported connection resets are isolated to a specific component.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, linux, python
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100