modelscope / modelscope/ms-swift
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Describe the bug
使用
IMAGE_MAX_TOKEN_NUM=128 CUDA_VISIBLE_DEVICES=0,1,2,3 swift deploy --model /Exp008/v0-20251231-163417/iter_0000600_hf --infer_backend vllm --port 8001 --api_key abc123 --served_model_name Exp008_step600 --vllm_tensor_parallel_size 4 --max_new_tokens 30000 --vllm_gpu_memory_utilization 0.8部署Qwen3-vl-32B-think进行推理,16并发推理,会遇到以下问题
[INFO:swift] Traceback (most recent call last): File "/root/training_env_sft/ms-swift/swift/llm/infer/deploy.py", line 198, in create_chat_completion res_or_gen = await self.infer_async(infer_request, request_config, template=self.template, **infer_kwargs) File "/root/training_env_sft/ms-swift/swift/llm/infer/infer_engine/vllm_engine.py", line 806, in infer_async return await self._infer_full_async(**kwargs) File "/root/training_env_sft/ms-swift/swift/llm/infer/infer_engine/vllm_engine.py", line 671, in _infer_full_async async for result in result_generator: File "/root/training_env_sft/.venv/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 370, in generate q = await self.add_request( File "/root/training_env_sft/.venv/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 276, in add_request raise EngineDeadError() vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
INFO: 28.18.40.30:47747 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
Your hardware and system info
这是我的环境信息:
vllm 0.11.0
torch 2.8.0
ms-swift 3.13.0.dev0
Additional context
期待您的回复
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with swift/llm/infer/deploy.py at create_chat_completion and swift/llm/infer/infer_engine/vllm_engine.py at infer_async and _infer_full_async. Reproduce the deployment command with the reported environment and inspect the missing EngineCore stack trace, including vllm/v1/engine/async_llm.py. Done means identifying the root cause of EngineDeadError under 16 concurrent requests and documenting or validating the fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100