vicuna-7b fastchat.serve.cli stops loading checkpoint shards in my google colab
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
I've been trying to get fastchat to work many times but i always get stuck here. I'm using a google colab with python 3.10.12 and i've followed the installation instructions.
i'm trying to run this:
!python -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.3
and here's the output:
023-06-24 16:34:14.704392: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Loading checkpoint shards: 50% 1/2 [01:23<01:23, 83.38s/it]^C
(i did not stop it)
not sure if it has to deal with trensorRT.
I've already tried with vicuna-7b-v1.3, vicuna-13b-v1.3 and fastchat-t5-3b-v1.0 and they all end up with this output.
I've also tried with only using cpu and still the same.
Thanks
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported command, python -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.3, in the stated Google Colab and Python 3.10.12 environment. Start from the fastchat.serve.cli entry point and determine whether the TensorRT warning or checkpoint loading is responsible; done means identifying the cause and confirming that the model loads successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100