tool_choice: "required" crashes EngineCore with SYCL device exception; "auto" works fine with identical request
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
**Environment:**
- Hardware: Intel Arc Pro B70 (Battlemage), 32GB
- OS: Bazzite (Fedora Atomic)
- Affected images: `intel/llm-scaler-vllm:0.14.0-b8.2.1` and `0.14.0-b8.3.1` (vLLM `0.14.1.dev0+gb17039bcc.d20260605`)
- Confirmed working image: `intel/vllm:0.10.2-xpu` (vLLM `0.1.dev9453+g1babc91fe.d20251028`) — identical request shapes run clean on this older build
- Models tested: `Qwen/Qwen3-8B` and `unsloth/Meta-Llama-3.1-8B-Instruct` — both affected
- Launch flags: `--enforce-eager --enable-auto-tool-choice --tool-call-parser hermes` (also reproduced with `llama3_json` parser on the Llama model)
- Container started via `--entrypoint bash -c "source /opt/intel/oneapi/setvars.sh --force && exec vllm serve ..."` — confirmed `setvars.sh` runs successfully every time (oneAPI init banner present in full in every log, including the crash logs below). Aware of `KNOWN_ISSUES.md` #3 regarding `--enable-auto-tool-choice` + missing `setvars.sh` sourcing causing OOM — that's not what's happening here; this is a distinct, reliably reproducible crash specific to `tool_choice: "required"`.
**Summary:**
Any chat completion request with `tool_choice: "required"` reliably crashes the EngineCore process with an unhandled `sycl::_V1::exception: No device of requested type available`, killing the entire vLLM server (requires a restart). The identical request with `tool_choice: "auto"` — same tools, same messages — completes successfully every time. This reproduces 100% of the time, isolated down to a single tool with zero parameters and a one-word prompt.
**Minimal repro (crashes):**
```json
{"model": "qwen3-8b", "messages": [{"role": "user", "content": "hi"}], "tools": [{"type": "function", "function": {"name": "noop", "parameters": {"type": "object", "properties": {}}}}], "tool_choice": "required"}
```
**Same request, only `tool_choice` changed (succeeds):**
```json
{"model": "qwen3-8b", "messages": [{"role": "user", "content": "hi"}], "tools": [{"type": "function", "function": {"name": "noop", "parameters": {"type": "object", "properties": {}}}}], "tool_choice": "auto"}
```
**Server-side traceback (truncated):**
```
terminate called after throwing an instance of 'sycl::_V1::exception'
what(): No device of requested type available. ...
ERROR [core_client.py:610] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.
ERROR [async_llm.py:546] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue.
```
**Client-side response:**
```json
{"error":{"message":"EngineCore encountered an issue. See stack trace (above) for the root cause.","type":"Internal Server Error","param":null,"code":500}}
```
**Isolation steps taken:**
1. Confirmed crash with a real production payload (13 tools, full conversation history, `tool_choice: "required"`) — crashed.
2. Same payload, only `tool_choice` changed to `"auto"` — succeeded.
3. Reverted to `"required"`, but trimmed to a single zero-parameter tool and a short prompt — crashed.
4. Stripped to the absolute minimal repro above (one tool, one-word prompt) — crashed identically.
**Notes:**
- Crash happens specifically when an actual request is processed — model load, KV cache profiling, and the internal warmup forward pass all succeed normally beforehand.
- Suspect this is in the guided-decoding/structured-output path specifically exercised by forcing a tool selection (vs. `"auto"`, which leaves a free-text branch in the grammar) — possibly XPU-specific given the SYCL device error.
- Isolated across multiple sessions with real production traffic (an AI radio DJ agent making real tool calls) and confirmed with the minimal synthetic payload above, on two different models and two different image tags.
Contributor guide
Assessment
This issue has not been assessed yet.