Issues noticed when trying https://github.com/intel/llm-scaler/releases/tag/vllm-0.14.0-b8.3
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
I noticed when I tested the new 0.14.0-b8.3 on Qwen/Qwen3.6-35b-a3b moe
I used llm scaler vllm docker image 0.14.0-b8.3 on ubuntu with 2x b70 with tags like
--quantization fp8
--max-model-len 256000
--gpu-memory-utilization 0.92
--dtype=bfloat16
The custom SYCL/ESIMD MoE kernel (/tmp/custom-esimd-kernels-vllm/csrc/moe_batch/moe.sycl) only supports float16, not bfloat16.
--kv_cache_dtype fp8
Starts serving but crashes. The page_attn_decode kernel in eagle.sycl:341 seems to rejects the fp8 KV cache.
--default-chat-template-kwargs '{"enable_thinking": true}'
--reasoning-parser qwen3
Is ignored and the model is thinking out loud, both when added or removed with true/false statement.
Its start output with: Here's a thinking process: and ends reasoning content with something like: Proceed. Output matches the refined response.✅
I have tried passing through --trust-remote-code and/or --chat-template /llm/models/Qwen3.6-35B-A3B/chat_template.jinja and I still cant get it to separate reasoning content from content.
Looks like the output might not be generating < think > < /think > correctly. The problem seems to be that the reasoning parser plugin isn't loaded in this V1 engine build (reasoning_parser_plugin=''), so reasoning_content stays None.
Contributor guide
Assessment
This issue has not been assessed yet.