lm-sys / lm-sys/FastChat

load new models to fastchat

Open
#2,610 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
PR merge metrics
No merged PRs in 30d

Description

I have been using a new model that claimed to be built upon fastchat,
I have downloaded all these models file and put them into the folder but they still won't work out
https://huggingface.co/jamessyx/pathasst_caption_tool/tree/main

I have succesfully run the fastchat default model,
When I run the following
python3 -m fastchat.serve.cli --model /location-to-/pathasst_caption_tool --device mps
I have errors like,
Traceback (most recent call last):
File "/opt/homebrew/lib/python3.11/site-packages/peft/config.py", line 100, in from_pretrained
config_file = hf_hub_download(
^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
validate_repo_id(arg_value)
File "/opt/homebrew/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/location/pathasst_caption_tool'. Use repo_type argument if needed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code

File "/Users/bidetime/Research/Projects/FastChat/fastchat/model/model_adapter.py", line 520, in load_model
config = PeftConfig.from_pretrained(model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/peft/config.py", line 104, in from_pretrained
raise ValueError(f"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'")
ValueError: Can't find 'adapter_config.json' at '/pathasst_caption_tool/'

I don't know where fastchat used to save the models under lmsys/ so it was difficult to find solutions for it. Please help.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the command python3 -m fastchat.serve.cli --model /location-to-/pathasst_caption_tool --device mps and read fastchat/model/model_adapter.py around line 520. Check how the CLI passes the local model path to PeftConfig.from_pretrained and compare it with the downloaded model files, especially adapter_config.json. Done means the supported loading behavior or the missing model-file requirement is documented or addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.