V1 engine breaks block size for gpt-oss-120b
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
Possible regression / changed requirement in V1 XPU path:
Previously, `--block-size 32` worked with this model/config. On the current Intel Scaler / V1 XPU path, the server starts successfully, loads the model, completes init, then fails on first request with:
RuntimeError: XeTLA ChunkPrefill FP8KV: only support block_size >= 64
This is surprising because the config is accepted at startup and only fails at execution time after a long model load/warmup.
Observed path:
- vLLM V1 engine
- XPU backend
- GPT-OSS-120B
- MXFP4
- chunked prefill enabled
- Flash Attention backend
- `--block-size 32`
Workaround:
- `--block-size 64` works
- Next test: `--block-size 32 --disable-chunked-prefill`
Request:
Please document this requirement, reject invalid `--block-size` values at startup when this backend path is selected, or allow the old behavior if there is a supported non-FP8KV/chunked-prefill path.
This is especially painful because the failure happens after the model is already loaded and the API server has started.
A startup validation error would save a lot of debugging time. And reboots because it wedges the GPUs
Contributor guide
Assessment
This issue has not been assessed yet.