Structured output silently dropped when a model declares the feature but its plugin does not implement it
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 20h 50m
- Merged PRs (30d)
- 586
Description
### Self Checks
- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.
### Dify version
1.16.1
### Cloud or Self Hosted
Self Hosted (Docker)
### Steps to reproduce
1. Use any model whose plugin YAML declares `structured-output` in its
`features` list while the plugin's `llm.py` has no corresponding
implementation.
A current, concrete instance is `claude-sonnet-4-6` in
`langgenius/anthropic` v0.3.26, filed separately as
langgenius/dify-official-plugins#3671.
2. Create an LLM node with that model, enable **Structured Output**, and define
any schema.
3. Run the node with a prompt that would naturally produce a different shape.
### ✔️ Expected Behavior
Structured output either takes effect, or fails visibly.
A capability a model declares but its plugin cannot deliver should not degrade
silently. Concretely, either the schema is enforced natively, or the request
falls back to prompt-based schema injection — the same treatment models that do
not declare the feature already receive.
### ❌ Actual Behavior
The schema is applied by neither the native path nor the prompt fallback, and
no warning appears in the editor. The request completes with HTTP 200 and
returns unconstrained content.
Two related causes.
### 1 — the native branch is taken on a declaration alone, and it excludes the fallback
`api/core/llm_generator/output_parser/structured_output.py`, lines 121-133
(verified identical in `1.16.1` and `main`):
```python
if model_schema.support_structure_output:
model_parameters = _handle_native_json_schema(
provider, model_schema, json_schema, model_parameters_with_json_schema, model_schema.parameter_rules
)
else:
_set_response_format(model_parameters_with_json_schema, model_schema.parameter_rules)
prompt_messages = _handle_prompt_based_schema(
prompt_messages=prompt_messages,
structured_output_schema=json_schema,
)
```
`support_structure_output` resolves purely from the model YAML's `features`
list. Nothing verifies that the provider plugin actually consumes the schema.
`_handle_native_json_schema`, lines 200-225, sets
`model_parameters["json_schema"]`, then sets `response_format` only if the model
declares a `response_format` parameter rule listing `json_schema`:
```python
model_parameters["json_schema"] = json.dumps(schema_json, ensure_ascii=False)
for rule in rules:
if rule.name == "response_format" and ResponseFormat.JSON_SCHEMA in rule.options:
model_parameters["response_format"] = ResponseFormat.JSON_SCHEMA
```
For a provider whose API has no `response_format` concept, no such rule exists
and nothing is set. If that provider's plugin also does not read `json_schema`,
the schema is dropped — and the prompt fallback has already been excluded by
the branch above.
So declaring the feature can produce a **worse** outcome than not declaring it:
a model omitting `structured-output` takes the else branch and does receive
prompt-based schema guidance.
### 2 — the editor indicator and the runtime read different sources
The LLM node's structured-output indicator resolves from served model metadata,
while the runtime branches on the plugin's YAML declaration. Where the two
disagree, the editor shows the permissive answer.
Observed with `langgenius/anthropic` v0.3.26:
| Model | Served metadata | Plugin YAML | Editor shows |
|---|---|---|---|
| `claude-sonnet-5` | `structured-output` present | absent | no warning |
| `claude-sonnet-4-6` | present | present | no warning |
| `claude-sonnet-4-5-20250929` | absent | absent | downgrade warning |
For `claude-sonnet-5` the editor shows no warning while the runtime takes the
prompt-injection path. Since that warning is the only signal most users have,
its absence currently means "one of two sources reports support" rather than
"this is enforced".
### Suggested direction
Either of these closes the silent-degradation path:
- Fall back to prompt-based schema injection when a native structured-output
request yields no structured output, rather than returning unconstrained
content; or
- Validate the model's declaration against the plugin's capability at load
time, and surface a warning when the two disagree.
The second is cheaper and would at least make the state visible.
Contributor guide
Research direction
Start in api/core/llm_generator/output_parser/structured_output.py, especially the native and prompt-based branches at the cited lines. Trace how support_structure_output, the model YAML features, served metadata, and plugin llm.py capabilities reach the runtime and editor. Compare a declared-but-unimplemented provider with one that lacks the declaration. Done means the schema is enforced or visibly falls back, and mismatches no longer silently allow unconstrained output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100