langgenius / langgenius/dify

Structured output silently dropped when a model declares the feature but its plugin does not implement it

Open
#40,907 5 comments 1 reaction 0 assignees View on GitHub
🐞 bug
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
20h 50m
Merged PRs (30d)
586

Description

### Self Checks

- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.

### Dify version

1.16.1

### Cloud or Self Hosted

Self Hosted (Docker)

### Steps to reproduce

1. Use any model whose plugin YAML declares `structured-output` in its
`features` list while the plugin's `llm.py` has no corresponding
implementation.

A current, concrete instance is `claude-sonnet-4-6` in
`langgenius/anthropic` v0.3.26, filed separately as
langgenius/dify-official-plugins#3671.

2. Create an LLM node with that model, enable **Structured Output**, and define
any schema.
3. Run the node with a prompt that would naturally produce a different shape.

### ✔️ Expected Behavior

Structured output either takes effect, or fails visibly.

A capability a model declares but its plugin cannot deliver should not degrade
silently. Concretely, either the schema is enforced natively, or the request
falls back to prompt-based schema injection — the same treatment models that do
not declare the feature already receive.

### ❌ Actual Behavior

The schema is applied by neither the native path nor the prompt fallback, and
no warning appears in the editor. The request completes with HTTP 200 and
returns unconstrained content.

Two related causes.

### 1 — the native branch is taken on a declaration alone, and it excludes the fallback

`api/core/llm_generator/output_parser/structured_output.py`, lines 121-133
(verified identical in `1.16.1` and `main`):

```python
if model_schema.support_structure_output:
model_parameters = _handle_native_json_schema(
provider, model_schema, json_schema, model_parameters_with_json_schema, model_schema.parameter_rules
)
else:
_set_response_format(model_parameters_with_json_schema, model_schema.parameter_rules)
prompt_messages = _handle_prompt_based_schema(
prompt_messages=prompt_messages,
structured_output_schema=json_schema,
)
```

`support_structure_output` resolves purely from the model YAML's `features`
list. Nothing verifies that the provider plugin actually consumes the schema.

`_handle_native_json_schema`, lines 200-225, sets
`model_parameters["json_schema"]`, then sets `response_format` only if the model
declares a `response_format` parameter rule listing `json_schema`:

```python
model_parameters["json_schema"] = json.dumps(schema_json, ensure_ascii=False)

for rule in rules:
if rule.name == "response_format" and ResponseFormat.JSON_SCHEMA in rule.options:
model_parameters["response_format"] = ResponseFormat.JSON_SCHEMA
```

For a provider whose API has no `response_format` concept, no such rule exists
and nothing is set. If that provider's plugin also does not read `json_schema`,
the schema is dropped — and the prompt fallback has already been excluded by
the branch above.

So declaring the feature can produce a **worse** outcome than not declaring it:
a model omitting `structured-output` takes the else branch and does receive
prompt-based schema guidance.

### 2 — the editor indicator and the runtime read different sources

The LLM node's structured-output indicator resolves from served model metadata,
while the runtime branches on the plugin's YAML declaration. Where the two
disagree, the editor shows the permissive answer.

Observed with `langgenius/anthropic` v0.3.26:

| Model | Served metadata | Plugin YAML | Editor shows |
|---|---|---|---|
| `claude-sonnet-5` | `structured-output` present | absent | no warning |
| `claude-sonnet-4-6` | present | present | no warning |
| `claude-sonnet-4-5-20250929` | absent | absent | downgrade warning |

For `claude-sonnet-5` the editor shows no warning while the runtime takes the
prompt-injection path. Since that warning is the only signal most users have,
its absence currently means "one of two sources reports support" rather than
"this is enforced".

### Suggested direction

Either of these closes the silent-degradation path:

- Fall back to prompt-based schema injection when a native structured-output
request yields no structured output, rather than returning unconstrained
content; or
- Validate the model's declaration against the plugin's capability at load
time, and surface a warning when the two disagree.

The second is cheaper and would at least make the state visible.

Contributor guide

Open the contributing guide

Research direction

Start in api/core/llm_generator/output_parser/structured_output.py, especially the native and prompt-based branches at the cited lines. Trace how support_structure_output, the model YAML features, served metadata, and plugin llm.py capabilities reach the runtime and editor. Compare a declared-but-unimplemented provider with one that lacks the declaration. Done means the schema is enforced or visibly falls back, and mismatches no longer silently allow unconstrained output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.