InternLM / InternLM/lmdeploy

[Deploy Error] ValueError: If `eos_token_id` is defined, make sure that `pad_token_id` is defined.

Open
#522 3 comments 0 reactions 1 assignee Claimed by @grimoire View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

(xtuner) ➜ xtuner git:(main) ✗ python -m lmdeploy.pytorch.chat work_dirs/internlm_20b_qlora_oasst1_512_e3/hf_merge \
--max_new_tokens 256 \
--temperture 0.8 \
--top_p 0.95 \
--seed 0
[2023-10-01 05:23:35,375] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Worker 0 logging to chat.log
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████| 22/22 [00:14<00:00, 1.51it/s]
Traceback (most recent call last):
File "miniconda/envs/xtuner/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "miniconda/envs/xtuner/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "miniconda/envs/xtuner/lib/python3.10/site-packages/lmdeploy/pytorch/chat.py", line 212, in
cli()
File "miniconda/envs/xtuner/lib/python3.10/site-packages/lmdeploy/pytorch/chat.py", line 208, in cli
fire.Fire(main)
File "miniconda/envs/xtuner/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "miniconda/envs/xtuner/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "miniconda/envs/xtuner/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "miniconda/envs/xtuner/lib/python3.10/site-packages/lmdeploy/pytorch/chat.py", line 158, in main
model.generate(torch.tensor([[6]], device=get_local_rank()), warmup_config)
File "miniconda/envs/xtuner/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "miniconda/envs/xtuner/lib/python3.10/site-packages/transformers/generation/utils.py", line 1648, in generate
return self.sample(
File "miniconda/envs/xtuner/lib/python3.10/site-packages/transformers/generation/utils.py", line 2771, in sample
raise ValueError("If `eos_token_id` is defined, make sure that `pad_token_id` is defined.")
ValueError: If `eos_token_id` is defined, make sure that `pad_token_id` is defined.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.