trtllm-serve accepts `response_format` and never enforces it

Description

trtllm-serve accepts an OpenAI response_format request, returns HTTP 200, and produces output that does not satisfy the requested format with no error and no warning. This is only a silent correctness bug.
Server stays healthy and all other requests are served normally. It occurs in two ways: (1) when the server is started without a guided_decoding_backend, the constraint is dropped entirely and the request is answered with free-form text; and (2) even with a backend configured, a type: "json_schema" request has its declared schema dropped, because the OpenAI wrapper object ({"name": ..., "schema": ...}) is passed to the grammar backend instead of the inner schema — so the output is unconstrained JSON rather than matching the schema (e.g. a request for {"x": } returns a bare 100). In both cases a client relying on structured output silently receives unstructured or off-schema text.

Environment

• TensorRT-LLM 1.2.1 (release container nvcr.io/nvidia/tensorrt-llm/release:1.2.1), PyTorch backend, single GPU (RTX A6000), TLLM_WORKER_USE_SINGLE_PROCESS=1.
• trtllm-serve <model> OpenAI server, /v1/chat/completions.
• Models: Tried on both Qwen/Qwen2.5-0.5B-Instruct, Qwen/Qwen2.5-1.5B-Instruct (bf16).
• xgrammar is present in the container.
• CUDA 13.1 / cuDNN 9.17.0, torch 2.10.0a0

Reproduce

Self-contained; needs only the release container and a small model.

# Bug 1: no backend
trtllm-serve Qwen/Qwen2.5-0.5B-Instruct --host 0.0.0.0 --port 8000
curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model":"Qwen/Qwen2.5-0.5B-Instruct",
  "messages":[{"role":"user","content":"Give me a point with integer x."}],
  "max_tokens":48,"temperature":0.0,
  "response_format":{"type":"json_schema","json_schema":{"name":"point","schema":{"type":"object","properties":{"x":{"type":"integer"}},"required":["x"]}}}}'
# -> 200, prose, no warning

# Bug 2: with backend
printf 'guided_decoding_backend: xgrammar\n' > guided.yaml
trtllm-serve Qwen/Qwen2.5-0.5B-Instruct --extra_llm_api_options guided.yaml --host 0.0.0.0 --port 8000
# same request -> content "100" (bare int; object schema not enforced)
# send json_schema set to the schema object directly (no name/schema wrapper) -> output locked to an object

More Details:

There are two distinct defects in the response_format to guided-decoding path.

tart trtllm-serve normally (no --extra_llm_api_options), send a chat request with response_format: {"type": "json_schema", ...}:

{"role":"assistant","content":"I'm sorry, but I can't provide a point with an integer x because I don't have access to ...","finish_reason":"length"}

HTTP 200, free prose, and nothing in the server log mentions guided decoding. The field was accepted and had no effect.

Because serve/openai_protocol.py always converts response_format into GuidedDecodingParams per request, but the engine only builds a guided decoder when a backend was configured — _torch/pyexecutor/_util.py:

feature_status["guided_decoding"] = llm_args.guided_decoding_backend is not None

With no backend, self.guided_decoder is None, every enforcement site (if self.guided_decoder is not None: in py_executor.py / model_engine.py) is skipped, and the constraint is dropped. serve/openai_server.py never checks whether guided decoding is available before accepting the request (a grep forguided_decoding there returns nothing), so there is no 4xx and no warning. A client asking for structured output gets unstructured output and a 200.

Expected: either engage a default grammar backend, or reject / warn when a request needs guided decoding and none is configured — not accept-and-ignore.

Bug 2, Even if we have a backend configured, type: "json_schema" passes the OpenAI wrapper to the grammar, so the declared schema is never enforced

Start with a backend:

# guided.yaml, passed as: trtllm-serve <model> --extra_llm_api_options guided.yaml guided_decoding_backend: xgrammar

Send the standard OpenAI structured-output request — an object with a required integer x:

"response_format": {"type":"json_schema",
  "json_schema":{"name":"point","schema":{"type":"object",
    "properties":{"x":{"type":"integer"}},"required":["x"],"additionalProperties":false}}}

Result: content: "100", finish_reason: stop. The model completed a bare integer, not the required object. The object schema was not enforced.

Same server, same xgrammar, same {x:integer} schema, but sending the schema where TRT-LLM actually reads it (see cause) instead locks the output into an object:

how the schema is sent	output	finish
json_schema: {"name":"point","schema":{…}} (OpenAI standard)	100	stop
json_schema: {…the schema directly…}	{ … (object-locked, then hit max_tokens)	length
type:"json", schema:{…}	{ … (object-locked, then hit max_tokens)	length

The wrapper case can produce and finish on a bare integer; the two direct-schema cases cannot emit anything but a JSON object. So the wrapper is not the object constraint — it is being accepted as a schema that constrains nothing.

This seems to be fromserve/openai_protocol.py, _response_format_to_guided_decoding_params:

elif response_format.type == "json_schema":
    ...
    return GuidedDecodingParams(json=response_format.json_schema)   # <-- the wrapper

response_format.json_schema is the OpenAI wrapper {"name": ..., "schema": {…}}, not a JSON Schema. As a schema it has no top-level type/properties, so xgrammar constrains to any JSON — which is why a bare 100 is allowed. It should pass the inner schema, e.g. response_format.json_schema["schema"]. (The type:"json" branch on the line above already uses the direct response_format.schema, and that one locks to an object.)

Expected: the declared JSON Schema is enforced; a request for {x:integer} cannot return 100.

See the above for reproduction

Start in serve/openai_protocol.py at _response_format_to_guided_decoding_params, then trace the request through serve/openai_server.py and the guided-decoding checks in _torch/pyexecutor/py_executor.py and model_engine.py. Reproduce both supplied trtllm-serve cases with and without a configured backend. Done means json_schema requests enforce the declared schema and requests without a backend are rejected or warned instead of silently returning unconstrained output.