PaddlePaddle / PaddlePaddle/PaddleX
PaddleX模型剪裁报错:parallel_for failed: no kernel image is available for execution on the device
@cuicheng01 is already working on this.
Since Feb 26, 2024.
- Dominant language
- Python
- Stars
- 6.3k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
问题类型:模型训练
问题描述
PaddleX 1.3.11 版本,模型训练没有问题,但是在模型剪裁的时候报错了。报错内容如下:
creating index...
index created!
Paddle Version: 1.8.5
PaddleX Version: 1.3.11
num_classes = 1
W0702 08:14:17.499222 211 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 86, Driver API Version: 11.2, Runtime API Version: 10.0
W0702 08:14:17.501857 211 device_context.cc:260] device: 0, cuDNN Version: 8.1.
W0702 08:14:22.019361 211 operator.cc:187] gaussian_random raises an exception thrust::system::system_error, parallel_for failed: no kernel image is available for execution on the device
/usr/local/lib/python3.7/dist-packages/paddle/fluid/executor.py:1070: UserWarning: The following exception is not an EOF exception.
"The following exception is not an EOF exception.")
Traceback (most recent call last):
File "model_prune_paddlex1.3.11.py", line 59, in
use_vdl=True)
File "/usr/local/lib/python3.7/dist-packages/paddlex/cv/models/yolo_v3.py", line 174, in train
early_stop_patience, resume_checkpoint, False)
File "/usr/local/lib/python3.7/dist-packages/paddlex/cv/models/ppyolo.py", line 354, in train
resume_checkpoint=resume_checkpoint)
File "/usr/local/lib/python3.7/dist-packages/paddlex/cv/models/base.py", line 247, in net_initialize
self.exe.run(startup_prog)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/executor.py", line 1071, in run
six.reraise(*sys.exc_info())
File "/usr/local/lib/python3.7/dist-packages/six.py", line 719, in reraise
raise value
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/executor.py", line 1066, in run
return_merged=return_merged)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/executor.py", line 1154, in _run_impl
use_program_cache=use_program_cache)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/executor.py", line 1229, in _run_program
fetch_var_name)
RuntimeError: parallel_for failed: no kernel image is available for execution on the device
各个工具的版本号如下:
paddle2onnx 0.6
paddlehub 2.1.0
paddlenlp 2.0.5
paddlepaddle-gpu 1.8.5.post107
paddleslim 1.1.1
paddlex 1.3.11
电脑显卡:RTX3090,系统:Ubuntu18.04
====================
请在这里描述您在使用过程中的问题,如模型训练出错,建议贴上模型训练代码,以便开发人员分析,并快速响应
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.