vicuna-7b-v1.5 err
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
My Env is:
Windows 10
Python 3.10.6
FastChat-0.2.25
When running worker Err:
(fchat) D:\ML2023\FastChat-0.2.25>python -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5
2023-08-25 16:19:29 | INFO | model_worker | args: Namespace(host='localhost', port=21002, worker_address='http://localhost:21002', controller_address='http://localhost:21001', model_path='lmsys/vicuna-7b-v1.5', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False)
2023-08-25 16:19:29 | INFO | model_worker | Loading the model ['vicuna-7b-v1.5'] on worker a8d76c1d ...
2023-08-25 16:19:29 | ERROR | stderr | Traceback (most recent call last):
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\runpy.py", line 196, in _run_module_as_main
2023-08-25 16:19:29 | ERROR | stderr | return _run_code(code, main_globals, None,
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\runpy.py", line 86, in _run_code
2023-08-25 16:19:29 | ERROR | stderr | exec(code, run_globals)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\serve\model_worker.py", line 509, in
2023-08-25 16:19:29 | ERROR | stderr | args, worker = create_model_worker()
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\serve\model_worker.py", line 486, in create_model_worker
2023-08-25 16:19:29 | ERROR | stderr | worker = ModelWorker(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\serve\model_worker.py", line 212, in init
2023-08-25 16:19:29 | ERROR | stderr | self.model, self.tokenizer = load_model(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\model\model_adapter.py", line 278, in load_model
2023-08-25 16:19:29 | ERROR | stderr | model, tokenizer = adapter.load_model(model_path, kwargs)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\ML2023\FastChat-0.2.25\fastchat\model\model_adapter.py", line 532, in load_model
2023-08-25 16:19:29 | ERROR | stderr | tokenizer = AutoTokenizer.from_pretrained(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 727, in from_pretrained
2023-08-25 16:19:29 | ERROR | stderr | return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\tokenization_utils_base.py", line 1854, in from_pretrained
2023-08-25 16:19:29 | ERROR | stderr | return cls._from_pretrained(
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\tokenization_utils_base.py", line 2017, in _from_pretrained
2023-08-25 16:19:29 | ERROR | stderr | tokenizer = cls(*init_inputs, **init_kwargs)
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\models\llama\tokenization_llama.py", line 156, in init
2023-08-25 16:19:29 | ERROR | stderr | self.sp_model = self.get_spm_processor()
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\models\llama\tokenization_llama.py", line 164, in get_spm_processor
2023-08-25 16:19:29 | ERROR | stderr | model_pb2 = import_protobuf()
2023-08-25 16:19:29 | ERROR | stderr | File "D:\Develop\anaconda3\envs\fchat\lib\site-packages\transformers\convert_slow_tokenizer.py", line 40, in import_protobuf
2023-08-25 16:19:29 | ERROR | stderr | return sentencepiece_model_pb2
2023-08-25 16:19:29 | ERROR | stderr | UnboundLocalError: local variable 'sentencepiece_model_pb2' referenced before assignment
any ideas could help? Thanks
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The failure is reported from fastchat/serve/model_worker.py and fastchat/model/model_adapter.py; start by reproducing the documented model_worker command in the stated Windows and Python environment. Trace the tokenizer-loading failure through the reported stack, and consider the issue done when the worker starts successfully with lmsys/vicuna-7b-v1.5.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100