OpenMOSS / OpenMOSS/MOSS

torch1.10.1,cuda11.3,推理时报错RuntimeError: CUDA error:no kernel image...是因为显存不够吗,3080显卡

Open
#183 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
12.3k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

python moss_cli_demo.py
Fetching 17 files: 100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 83787.51it/s]
Waiting for all devices to be ready, it may take a few minutes...
欢迎使用 MOSS 人工智能助手!输入内容即可进行对话。输入 clear 以清空对话历史,输入 stop 以终止对话。
<|Human|>: hello
Traceback (most recent call last):
File "moss_cli_demo.py", line 89, in
main()
File "moss_cli_demo.py", line 72, in main
outputs = model.generate(
File "/root/miniconda3/envs/moss2/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/envs/moss2/lib/python3.8/site-packages/transformers/generation/utils.py", line 1358, in generate
if pad_token_id is not None and torch.sum(inputs_tensor[:, -1] == pad_token_id) > 0:
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with python moss_cli_demo.py using torch 1.10.1, CUDA 11.3, and an RTX 3080. Start at moss_cli_demo.py lines 72 and 89, then inspect the reported transformers generation call and environment details. Done would require establishing the cause and documenting a verified compatible setup or a clear correction path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.