vllm-project / vllm-project/aibrix

upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refuse

Open
#1,449 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### 🐛 Describe the bug

```
root@benchmark-client:/vllm-workspace/benchmarks# curl http://10.249.253.10:80/v1/chat/completions -H "Content-Type: application/json" -H "routing-strategy: pd" -d '{
"model": "qwen3-8B",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "help me write a random generator in python"}
]
}'
upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

```

### Steps to Reproduce

1p1d vllm deployment
model is loading, send request before it's ready

### Expected behavior

should return json response

### Environment

nightly

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the request against the 1P1D vLLM deployment while the model is loading, then inspect the deployment and upstream connection behavior. Done means the same chat-completions request returns a JSON response instead of the connection-refused error once the model is ready.

Written by the indexing model from the issue text.

Assessment

Domain
api, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.