vllm worker becomes unresponsive after running for a long time.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
When I use three processes to start the controller, vllm worker, and openai api server within a Docker container, after an extended period of operation, the vllm worker becomes unresponsive to requests from the openai api server. Upon external inspection, I observed that the vllm worker process is still running and has not been terminated. However, I did not encounter this issue when starting the services using three separate containers. What do you think could be the reason for this occurrence?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the three-process setup for the controller, vllm worker, and OpenAI API server inside one Docker container, then compare it with the three separate-container setup. Inspect communication between those processes and determine why the worker stops responding while remaining alive; done means identifying a reproducible cause and documenting or validating a fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100