intel / intel/llm-scaler

V1 engine breaks block size for gpt-oss-120b

Open
#390 1 comment 1 reaction 1 assignee Claimed by @liu-shaojun View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

Possible regression / changed requirement in V1 XPU path:

Previously, `--block-size 32` worked with this model/config. On the current Intel Scaler / V1 XPU path, the server starts successfully, loads the model, completes init, then fails on first request with:

RuntimeError: XeTLA ChunkPrefill FP8KV: only support block_size >= 64

This is surprising because the config is accepted at startup and only fails at execution time after a long model load/warmup.

Observed path:
- vLLM V1 engine
- XPU backend
- GPT-OSS-120B
- MXFP4
- chunked prefill enabled
- Flash Attention backend
- `--block-size 32`

Workaround:
- `--block-size 64` works
- Next test: `--block-size 32 --disable-chunked-prefill`

Request:
Please document this requirement, reject invalid `--block-size` values at startup when this backend path is selected, or allow the old behavior if there is a supported non-FP8KV/chunked-prefill path.

This is especially painful because the failure happens after the model is already loaded and the API server has started.

A startup validation error would save a lot of debugging time. And reboots because it wedges the GPUs

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.