torch1.10.1,cuda11.3,推理时报错RuntimeError: CUDA error:no kernel image...是因为显存不够吗,3080显卡
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
python moss_cli_demo.py
Fetching 17 files: 100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 83787.51it/s]
Waiting for all devices to be ready, it may take a few minutes...
欢迎使用 MOSS 人工智能助手!输入内容即可进行对话。输入 clear 以清空对话历史,输入 stop 以终止对话。
<|Human|>: hello
Traceback (most recent call last):
File "moss_cli_demo.py", line 89, in
main()
File "moss_cli_demo.py", line 72, in main
outputs = model.generate(
File "/root/miniconda3/envs/moss2/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/envs/moss2/lib/python3.8/site-packages/transformers/generation/utils.py", line 1358, in generate
if pad_token_id is not None and torch.sum(inputs_tensor[:, -1] == pad_token_id) > 0:
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with python moss_cli_demo.py using torch 1.10.1, CUDA 11.3, and an RTX 3080. Start at moss_cli_demo.py lines 72 and 89, then inspect the reported transformers generation call and environment details. Done would require establishing the cause and documenting a verified compatible setup or a clear correction path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100