For debugging consider passing CUDA_LAUNCH_BLOCKING=1

Hi， great job!

I run the demo program on a single 4090 (24g) video memory, and it can be started, but when asking questions, it will report the following error:

欢迎使用 MOSS 人工智能助手！输入内容即可进行对话。输入 clear 以清空对话历史，输入 stop 以终止对话。
<|Human|>: 介绍自己
Traceback (most recent call last):
File "/media/glc/jack/GPT/MOSS-main/moss_cli_demo.py", line 89, in
main()
File "/media/glc/jack/GPT/MOSS-main/moss_cli_demo.py", line 72, in main
outputs = model.generate(
File "/home/glc/anaconda3/envs/gpt/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, **kwargs)
File "/home/glc/anaconda3/envs/gpt/lib/python3.8/site-packages/transformers/generation/utils.py", line 1358, in generate
if pad_token_id is not None and torch.sum(inputs_tensor[:, -1] == pad_token_id) > 0:
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Start with moss_cli_demo.py around main() and the model.generate call at line 72, then reproduce the single-4090 failure while checking the reported CUDA error. Done means identifying a supported fix or clearly documenting the compatible CUDA/device setup and verifying that the demo can answer a question.