help！ llama2-13b use vllm_worker with two A100(80g); the RayWoker throw an exceptions
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
run in k8s pod； resource request：
memory 200G
gpu：2
cpu 40
could somebody tell me how to handle this problem？
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the WechatIMG7121 screenshot and the vllm_worker/RayWorker setup described in the issue. Reproduce the two-A100 run in the Kubernetes pod while collecting the complete exception and resource configuration. Done means identifying the failure cause and documenting a verified resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- ai, cloud, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100