vllm-project / vllm-project/aibrix

NIXL error: Potentially invalid KV blocks for unrecognized request

Open
#1,772 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 19h
Merged PRs (30d)
98

Description

### 🐛 Describe the bug

During benchmark, prefillers complain: ERROR 11-19 10:12:21 [distributed/.../v1/nixl_connector.py:1068] Potentially invalid KV blocks for unrecognized request chatcmpl-129a69c8-cd1a-40e0-aa3e-94662fd8cc2b were retrieved by a decode worker. They may have expired.

The engine continues to work after the error report.

### Steps to Reproduce

1. Deploy vllm 2p2d tp2 for Qwen3-32B.
2. Allocate 1 RDMA nic per pod.
3. Schedule prefill pods and decode pods onto separate nodes.
4. Conduct benchmark:

```
TOKENIZER="/data01/models/Qwen3-32B"
MODEL="qwen3-32b"
HOST="${LB_EXTERNAL_IP}"
PORT="80"
REQ_RATE=1.0
INPUT_LEN=8000
OUTPUT_LEN=200

python3 benchmark_serving.py --port $PORT --host $HOST --seed $(date +%s) \
--model $MODEL \
--tokenizer $TOKENIZER \
--dataset-name random --random-input-len ${INPUT_LEN} --random-output-len ${OUTPUT_LEN} \
--num-prompts 200 --burstiness 100 --request-rate ${REQ_RATE} --metric-percentiles 95 \
--backend openai-chat --endpoint /v1/chat/completions --routing-strategy "pd" --ignore-eos
```

### Expected behavior

No error reported during benchmarking.

### Environment

* AIBrix 0.5.0
* VKE

Contributor guide

Open the contributing guide

Research direction

Start with v1/nixl_connector.py at line 1068 and reproduce the two-prefill/two-decode Qwen3-32B benchmark using the provided deployment and benchmark command. Trace why a decode worker retrieves KV blocks for an unrecognized request, then verify the benchmark completes without the reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.