intel / intel/llm-scaler

[Bug] OpenAI-compatible streaming tool_calls omit function.name after first chunk, breaking OpenCode with Qwen3.6-27B on llm-scaler-vllm 0.14.0-b8.3.2

Open
#516 0 comments 0 reactions 1 assignee Claimed by @gc-fu View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

## Summary

Using `intel/llm-scaler-vllm:0.14.0-b8.3.2` with `Qwen3.6-27B` on a single Intel Arc Pro B70, streaming tool calls from the OpenAI-compatible `/v1/chat/completions` endpoint are emitted in a way that breaks strict OpenAI-compatible clients such as OpenCode Desktop v1.17.15.

The first `tool_calls` delta includes `id`, `type`, and `function.name`, but follow-up deltas only include fragments of `function.arguments` and do not re-emit `function.name`. In some parser configurations I also observed `function.name: null` in follow-up chunks.

This causes OpenCode to fail intermittently with:

`Expected 'function.name' to be a string`

The issue is intermittent in OpenCode because it only happens on turns where the model actually emits streamed tool calls in multiple chunks.

## Environment

- Hardware:
- 1 × Intel Arc Pro B70 32GB
- Ryzen 7 9700X
- 64GB DDR5 RAM
- OS:
- Ubuntu 26.04
- Container / build:
- `intel/llm-scaler-vllm:0.14.0-b8.3.2`
- Model:
- `/llm/models/Qwen3.6-27B-int4-AutoRound`
- served as `Qwen3.6-27B`
- Client:
- OpenCode Desktop v1.17.15
- Reproducible both with:
- `--tool-call-parser qwen3_xml`
- `--tool-call-parser qwen3_coder`

## Server launch command

```bash
vllm serve /llm/models/Qwen3.6-27B-int4-AutoRound \
--served-model-name Qwen3.6-27B \
--port 8000 \
--host 0.0.0.0 \
--dtype float16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-util 0.94 \
--max-model-len 245760 \
--max-num-batched-tokens 8192 \
--swap-space 16 \
--block-size 32 \
--max-num-seqs 2 \
--allow-deprecated-quantization \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_xml \
--chat-template /llm/models/qwen3.6_chat_template.jinja \
--default-chat-template-kwargs '{"max_tool_response_chars": 15000, "max_tool_arg_chars": 2000}'
```

I also tested `--tool-call-parser qwen3_coder` and saw the same root problem, but with follow-up chunks omitting `function.name` instead of sending `"name": null`.

## Reproduction

```bash
curl -N http://127.0.0.1:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen3.6-27B",
"stream": true,
"messages": [
{
"role": "user",
"content": "Lee el archivo api/informes.php usando una herramienta."
}
],
"tools": [
{
"type": "function",
"function": {
"name": "read_file",
"description": "Lee un archivo",
"parameters": {
"type": "object",
"properties": {
"path": { "type": "string" }
},
"required": ["path"]
}
}
}
],
"tool_choice": "auto",
"temperature": 0
}'
```

## Observed streaming output with `--tool-call-parser qwen3_coder`

### First tool-call delta

```json
data: {"choices":[{"index":0,"delta":{"reasoning_content":null,"tool_calls":[{"id":"call_7b2e5a158a144cb4b6930b67","type":"function","index":0,"function":{"name":"read_file","arguments":""}}]},"finish_reason":null}]}
```

### Follow-up deltas only carry argument fragments and do not re-emit `function.name`

```json
data: {"choices":[{"index":0,"delta":{"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"{"}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"\"path\": \"api/informes.php\""}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"}"}}]},"finish_reason":null}]}
```

### Final chunk

```json
data: {"choices":[{"index":0,"delta":{"content":"","reasoning_content":null},"finish_reason":"tool_calls"}]}
data: [DONE]
```

## Observed streaming output with `--tool-call-parser qwen3_xml`

In that configuration, follow-up deltas included:

```json
"function":{"name":null,"arguments":"..."}
```

instead of omitting `name`.

## Expected behavior

For better compatibility with strict OpenAI-compatible clients, follow-up `tool_calls` deltas should continue to include:

- `id`
- `type`
- `function.name`

on every tool-call delta for the same `index`, not only on the first one.

## Actual behavior

The server emits only the initial `function.name` and later emits partial argument deltas without `function.name` (or with `name: null` depending on parser). OpenCode Desktop v1.17.15 then fails with:

`Expected 'function.name' to be a string`

## Additional notes

- This is not a pure OpenCode-side hallucination issue: the broken chunk pattern is directly observable with `curl`.
- The problem is intermittent in OpenCode because it only happens when a tool call is streamed in multiple chunks.
- Switching from `qwen3_xml` to `qwen3_coder` changes the exact chunk shape, but does not solve the compatibility issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.