vllm-project / vllm-project/aibrix
upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refuse
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
### 🐛 Describe the bug
```
root@benchmark-client:/vllm-workspace/benchmarks# curl http://10.249.253.10:80/v1/chat/completions -H "Content-Type: application/json" -H "routing-strategy: pd" -d '{
"model": "qwen3-8B",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "help me write a random generator in python"}
]
}'
upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
```
### Steps to Reproduce
1p1d vllm deployment
model is loading, send request before it's ready
### Expected behavior
should return json response
### Environment
nightly
Contributor guide
Research direction
Start by reproducing the request against the 1P1D vLLM deployment while the model is loading, then inspect the deployment and upstream connection behavior. Done means the same chat-completions request returns a JSON response instead of the connection-refused error once the model is ready.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100