Model worker keeps on registering and gets de registered
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
I have a kubernetes deployment. The model worker runs on a separate node and the controller/api-server run on a different pod. But the model worker keeps on registering and in the mean time the requests to api keep failing with error. {"object":"error","message":"Only allowed now, your model vicuna-7b-v1.1","code":40301}
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the Kubernetes setup with the model worker on a separate node from the controller and API server, then inspect the worker registration and API request behavior. Done means the worker remains registered and API requests no longer fail with the reported 40301 error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- backend-api-design, distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100