swift deploy cannot fully disable Qwen3 “thinking”; no way to pass vLLM --chat-template. enable_thinking=False ignored on vLLM 0.8.5.post1

Describe the bug

Summary

When serving Qwen3 via swift deploy (vLLM backend pinned to vLLM 0.8.5.post1), I cannot reliably disable “thinking”:
• enable_thinking=False is ignored when the reasoning parser is enabled (known limitation in vLLM ≤0.8.5).
• swift deploy doesn’t expose a way to forward vLLM’s --chat-template flag, so I can’t use a non-thinking chat template (qwen3_nonthinking.jinja) as recommended by Qwen’s deployment docs.
• Depending on the route, the server either always separates thinking vs. answer, or never exposes reasoning_content, and the per-request toggle doesn’t work.

Per Qwen docs: in vLLM 0.9.0 with the qwen3 reasoning parser, enable_thinking=False works. Also, you can force non-thinking with a custom chat template. With ms-swift today, neither option is achievable out of the box.

⸻

Environment
• ms-swift: following docs at https://swift.readthedocs.io/en/latest/Instruction/Inference-and-deployment.html
• vLLM: 0.8.5.post1 (as in the swift docs)
• Model: Qwen/Qwen3-8B (also reproduced with a local Qwen3 checkpoint)
• Server: swift deploy (OpenAI-compatible /v1 API), vLLM backend
• Client(s):
• OpenAI Python SDK (chat.completions)
• swift.llm client (InferClient, InferRequest, RequestConfig)

⸻

What I run

Server (Swift)

swift deploy
--model Qwen/Qwen3-8B
--stream true
--infer_backend vllm
--use_hf true
--vllm_tensor_parallel_size 4
--vllm_gpu_memory_utilization 0.9
--vllm_max_model_len 32768
--vllm_reasoning_parser qwen3

I would like to add a non-thinking template, but swift deploy does not accept a --vllm_chat_template (or similar) flag. The vLLM CLI equivalent would be:
vllm serve Qwen/Qwen3-8B --chat-template ./qwen3_nonthinking.jinja

Client (Swift client)

from swift.llm import InferRequest, InferClient, RequestConfig
engine = InferClient(host='127.0.0.1', port=8000)

Attempt 1: per-request toggle via RequestConfig

request_config = RequestConfig(
max_tokens=512, temperature=0,
# hoping ms-swift forwards this to OpenAI body
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)

resp = engine.infer([InferRequest(messages=[{"role":"user","content":"Explain what is opacity in chest xray?"}])],
request_config)
print(resp[0].choices[0].message.dict)

Also tried attaching via InferRequest(objects={"chat_template_kwargs": {"enable_thinking": False}}).

⸻

Expected behavior
• On vLLM ≥ 0.9.0 with --reasoning-parser qwen3:
enable_thinking=False should suppress thinking (no …), and the API should either omit reasoning_content or return it empty.
• On vLLM 0.8.5.post1 (if staying pinned):
Allow me to force non-thinking at serve time by passing a custom chat template, e.g. forward vLLM’s --chat-template ./qwen3_nonthinking.jinja via a swift deploy flag.

⸻

Actual behavior
• With --vllm_reasoning_parser qwen3 on vLLM 0.8.5.post1:
• The server continues to produce “thinking” traces.
• enable_thinking=False sent from the client is ignored (consistent with vLLM ≤ 0.8.5 limitation).
• Depending on the route:
• OpenAI /v1/chat/completions: sometimes splits reasoning_content vs content, but the toggle still doesn’t disable thinking.
• Swift /infer: returns a single response string and does not surface reasoning_content at all, so I can’t inspect or hide it cleanly.
• I cannot pass a non-thinking chat template because swift deploy has no flag to forward vLLM’s --chat-template.

⸻

Relevant docs
• Qwen vLLM docs (Thinking / Non-thinking modes):
https://qwen.readthedocs.io/en/latest/deployment/vllm.html#thinking-non-thinking-modes
“As of vLLM 0.8.5, enable_thinking=False is not compatible with this feature… This is resolved in vLLM 0.9.0 with the qwen3 reasoning parser.”
“To completely disable thinking, use a custom chat template… --chat-template ./qwen3_nonthinking.jinja”
• ms-swift deployment docs (currently indicating vLLM 0.8.5.post1).

⸻

Steps to reproduce
1. Start swift deploy as shown above with --vllm_reasoning_parser qwen3.
2. From either the OpenAI SDK or swift.llm client, send:

extra_body={"chat_template_kwargs": {"enable_thinking": False}}

3.	Observe that the model still thinks (and/or the response schema doesn’t let you suppress/exclude reasoning cleanly).

Attempt to pass a non-thinking chat template from swift deploy — there is no supported flag, so this path is blocked.

⸻

Workarounds (not ideal)
• Bypass Swift and run vLLM directly:

vllm serve Qwen/Qwen3-8B --reasoning-parser qwen3 --chat-template ./qwen3_nonthinking.jinja

This immediately solves the problem, but loses Swift’s deployment ergonomics.

•	Strip <think>…</think> client-side with a regex — brittle and wastes tokens.

⸻

Requests / Proposed fixes
1. Expose vLLM --chat-template via swift deploy (e.g., --vllm_chat_template ), so we can enforce a non-thinking template when needed.
2. Upgrade/allow vLLM ≥ 0.9.0 in ms-swift deployments, and document that enable_thinking=False works with --vllm_reasoning_parser qwen3.
3. Ensure the Swift client (InferClient) has a clear, documented way to forward chat_template_kwargs (e.g., RequestConfig.extra_body and/or InferRequest.objects) to the OpenAI body.
4. Clarify the difference between the /infer route and the OpenAI /v1 route, and how/when reasoning_content is surfaced or suppressed.

Happy to provide more logs or test a PR if helpful.

Your hardware and system info
=== System Info ===
Linux qrnd22.l40.sr.internal.qure.ai 5.15.0-1083-nvidia #84-Ubuntu SMP Mon Jul 14 22:35:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

=== OS Release ===
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.5 LTS"
PRETTY_NAME="Ubuntu 22.04.5 LTS"

=== Python & Torch ===
Python 3.12.11
Torch version: 2.8.0+cu128
CUDA (built): 12.8
CUDA available: True
GPU 0: NVIDIA L40S
GPU 1: NVIDIA L40S
GPU 2: NVIDIA L40S
GPU 3: NVIDIA L40S

=== GCC / GLIBC ===
gcc (Ubuntu 11.4.0-1ubuntu1~22.04.2) 11.4.0
ldd (Ubuntu GLIBC 2.35-0ubuntu3.10) 2.35

=== Installed Packages (vllm, swift, torch) ===
Name: vllm
Version: 0.10.2rc2.dev89+gf94619747.cu129
Name: torch
Version: 2.8.0+cu128

ADDITION CONTEXT

the code i use to deploy

swift deploy \
    --model Qwen/Qwen3-8B \
    --stream true \
    --infer_backend vllm \
    --use_hf true \
    --vllm_tensor_parallel_size 4 \
    --vllm_gpu_memory_utilization 0.9 \
    --vllm_max_model_len 32768 \
    --vllm_reasoning_parser qwen3 \
    #--vllm_chat_template qwen3_nonthinking.jinja

Start at the swift deploy entry point and its vLLM argument forwarding, then trace InferClient and RequestConfig handling of extra_body across the /infer and OpenAI /v1 routes. Reproduce with Qwen/Qwen3-8B, qwen3 reasoning-parser settings, and the proposed qwen3_nonthinking.jinja template. Done means the template can be passed at deployment, request options are forwarded consistently, and the route behavior is documented or tested.