vllm_worker will be hung after chating a while
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
After vllm_worker for fastchat is started and chat for a while, the message responded by fastchat will be hung and then the screen showed "NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE".
The log of vllm_worker shows request is aborted.
Don't know why.
Does anyone knows the root cause of this issue or how to resolve it?
The web server log:
The vllm worker log:
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the hang after sustained chatting with the vllm_worker for FastChat, then inspect the web server and vllm worker logs around the aborted request. Done would mean identifying the cause of the aborted request and preventing the worker from hanging or returning the high-traffic network error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100