DeepSeek-V4: OpenAI `tools` field reorders the system prompt after the tool schemas

Component: tensorrt_llm/tokenizer/deepseek_v4/tokenizer.py
Version: TensorRT-LLM 1.3.0rc21
Verified against: NVIDIA/TensorRT-LLM main —
tokenizer/deepseek_v4/tokenizer.py is byte-identical to 1.3.0rc21; the
messages.insert(0, ...) at lines 441-442 is present on main. No existing
issue/PR found addressing this.
Severity: Medium — affects every /v1/chat/completions request that has both
a system message and tools

Summary

DeepseekV4Tokenizer.apply_chat_template handles the OpenAI tools argument by
inserting a synthetic {"role": "system", "tools": tools} message at index
0. The DeepSeek-V4 reference format attaches tools to the existing system
message. When a request has both, the user's system prompt is emitted after
the tool schemas instead of before, with no separator between them and a stray
\n\n after BOS.

Evidence

Reference implementation is encoding/encoding_dsv4.py, shipped inside the
DeepSeek-V4-Flash checkpoint. Comparing renderings of the same conversation
(system prompt "SYSTEM_PROMPT_HERE" + one tool + one user turn):

reference:  <BOS>SYSTEM_PROMPT_HERE\n\n## Tools\n...invoke tool calls.\n<｜User｜>Q<｜Assistant｜><think>
TRT-LLM:    <BOS>\n\n## Tools\n...invoke tool calls.\nSYSTEM_PROMPT_HERE<｜User｜>Q<｜Assistant｜><think>

Both strings are 1108 chars with an identical character multiset — it is a pure
transposition. The system prompt moves from offset 21 to offset 1061, landing
glued to the trailing \n of the tools block.
With reasoning_effort: "max" the effort preamble still lands first, so only
the system/tools pair is transposed.

Scope

• Affected: requests where tools are passed via the OpenAI tools field
  and a system message is present. This is the normal serving path:
  openai_server.py builds tool_dicts and passes it to
  async_apply_chat_template(tools=...) then
  inputs/utils.py:658-663 then tokenizer.apply_chat_template(messages, tools=tools).
• Not affected: no system message (verified identical to reference);
  developer role instead of system (identical); tools attached directly to a
  message dict rather than passed as the tools argument.
  The last case is why the checkpoint's own gold vector test_output_1.txt passes
  — it attaches tools to messages[0] and never exercises the tools= path.

Root cause

tensorrt_llm/tokenizer/deepseek_v4/tokenizer.py:441-442:

conversation = kwargs.get("conversation", messages)
messages = list(conversation)
if tools:
    messages.insert(0, {"role": "system", "tools": tools})   # always a NEW message

The renderer's system branch (tokenizer.py:307-312) emits
content then "\n\n" + render_tools(tools). With a synthetic message, index 0
has empty content, so it renders as "" + "\n\n" + <tools>, and the real system
message renders separately at index 1 — after it.

Suggested fix

Merge into a leading system message when one exists:

messages = list(conversation)
if tools:
    if messages and messages[0].get("role") == "system":
        messages[0] = {**messages[0], "tools": tools}
    else:
        messages.insert(0, {"role": "system", "tools": tools})

Verified against the checkpoint reference:

case	current	proposed
system + tools	mismatch	matches
no system, tools	matches	matches (no regression)
developer + tools	matches	matches (no regression)

Suggested regression test

The existing gold vectors do not cover the tools= argument path. Add a case
that passes tools via the argument alongside a system message, asserting
equality with the reference encoding.

Separately: the rest of the port is exact

For the record, the remainder of tokenizer.py matches
encoding_dsv4.py byte-for-byte. Verified:

• all 4 shipped gold vectors (tests/test_output_{1..4}.txt) reproduce exactly;
• a 36-case sweep of
  {basic, multiturn, no-system} x {chat, thinking} x {None, high, max} x drop_thinking{T,F}
  is byte-identical to the reference.
  Two intentional and harmless divergences:
• reasoning_effort validation: the reference asserts membership in
  ['max', None, 'high']; TRT-LLM silently coerces anything else to None.
  Reasonable for a server, but a typo'd value degrades quietly. Note only
  "max" has any effect — "high" is accepted but never read, in both
  implementations.
• TRT-LLM accepts reasoning as an alias for reasoning_content on input
  (tokenizer.py:296). This is correct — the server emits both fields in
  responses (serve/openai_protocol.py:681-682), so accepting both closes the
  round-trip. Strict superset of the reference.

Start in tensorrt_llm/tokenizer/deepseek_v4/tokenizer.py around lines 441-442 and inspect the system rendering branch at lines 307-312. Compare the tools= path with encoding/encoding_dsv4.py and the existing tests/test_output_{1..4}.txt gold vectors. Done means a system message followed by tools renders in reference order, while the no-system and developer cases remain unchanged, with a regression test covering tools passed as an argument.