failed to run FastChat with stable lm-zephyr-3b
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
nohup python -m fastchat.serve.controller --host 0.0.0.0 &
nohup python -m fastchat.serve.model_worker --model-path /Volumes/256/stablelm-zephyr-3b --host 0.0.0.0 --device mps &
sleep 40
nohup python -m fastchat.serve.gradio_web_server --host 0.0.0.0 &
Start fast chat without any issue.
But we use webui, I got the following error:
NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.
(Attention mask should be of size (1, 1, 1, 210), but is torch.Size([1, 1, 1, 1]))
(error_code: 50001)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure using the listed controller, model_worker, and gradio_web_server commands with the stablelm-zephyr-3b model on MPS. Start at the web UI request path and the reported attention-mask error; done means the model can generate through the web UI without the 50001 error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100