lm-sys / lm-sys/FastChat

vicuna-7b-v1.5 err

Open
#2,312 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
PR merge metrics
No merged PRs in 30d

Description

My Env is:
Windows 10
Python 3.10.6
FastChat-0.2.25
When running worker Err:
(fchat) D:\ML2023\FastChat-0.2.25>python -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5
2023-08-25 16:19:29 | INFO | model_worker | args: Namespace(host='localhost', port=21002, worker_address='http://localhost:21002', controller_address='http://localhost:21001', model_path='lmsys/vicuna-7b-v1.5', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False)
2023-08-25 16:19:29 | INFO | model_worker | Loading the model ['vicuna-7b-v1.5'] on worker a8d76c1d ...
2023-08-25 16:19:29 | ERROR | stderr | Traceback (most recent call last):
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\runpy.py", line 196, in _run_module_as_main
2023-08-25 16:19:29 | ERROR | stderr | return _run_code(code, main_globals, None,
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\runpy.py", line 86, in _run_code
2023-08-25 16:19:29 | ERROR | stderr | exec(code, run_globals)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\serve\model_worker.py", line 509, in
2023-08-25 16:19:29 | ERROR | stderr | args, worker = create_model_worker()
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\serve\model_worker.py", line 486, in create_model_worker
2023-08-25 16:19:29 | ERROR | stderr | worker = ModelWorker(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\serve\model_worker.py", line 212, in init
2023-08-25 16:19:29 | ERROR | stderr | self.model, self.tokenizer = load_model(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\model\model_adapter.py", line 278, in load_model
2023-08-25 16:19:29 | ERROR | stderr | model, tokenizer = adapter.load_model(model_path, kwargs)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\model\model_adapter.py", line 532, in load_model
2023-08-25 16:19:29 | ERROR | stderr | tokenizer = AutoTokenizer.from_pretrained(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 727, in from_pretrained
2023-08-25 16:19:29 | ERROR | stderr | return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\tokenization_utils_base.py", line 1854, in from_pretrained
2023-08-25 16:19:29 | ERROR | stderr | return cls._from_pretrained(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\tokenization_utils_base.py", line 2017, in _from_pretrained
2023-08-25 16:19:29 | ERROR | stderr | tokenizer = cls(*init_inputs, **init_kwargs)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\models\llama\tokenization_llama.py", line 156, in init
2023-08-25 16:19:29 | ERROR | stderr | self.sp_model = self.get_spm_processor()
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\models\llama\tokenization_llama.py", line 164, in get_spm_processor
2023-08-25 16:19:29 | ERROR | stderr | model_pb2 = import_protobuf()
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\convert_slow_tokenizer.py", line 40, in import_protobuf
2023-08-25 16:19:29 | ERROR | stderr | return sentencepiece_model_pb2
2023-08-25 16:19:29 | ERROR | stderr | UnboundLocalError: local variable 'sentencepiece_model_pb2' referenced before assignment

any ideas could help? Thanks

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The failure is reported from fastchat/serve/model_worker.py and fastchat/model/model_adapter.py; start by reproducing the documented model_worker command in the stated Windows and Python environment. Trace the tokenizer-loading failure through the reported stack, and consider the issue done when the worker starts successfully with lmsys/vicuna-7b-v1.5.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.