agentscope-ai / agentscope-ai/agentscope-java

[Bug]: OpenAIChatModel structuted output fails on DeepSeek/vLLM — supportsNativeStructuredOutput() returns true for unsupported backends

Open
#1,742 0 comments 0 reactions 0 assignees View on GitHub
area/core/model bug
Dominant language
Java
Stars
5.6k
Forks
1.3k
Avg merge
4d 12h
Merged PRs (30d)
77

Description

## Summary

`OpenAIChatModel.supportsNativeStructuredOutput()` hardcodes `true`, but many OpenAI-compatible backends (DeepSeek, vLLM, Ollama, SiliconFlow) do not support `response_format`. This causes `agent.call(msg, OutputClass.class)` to fail with HTTP 400 on those backends.

## Root Cause

`OpenAIChatModel.java:195`:

```java
@Override
public boolean supportsNativeStructuredOutput() {
return true; // ← assumes ALL OpenAI-compatible APIs support response_format
}
```

`ReActAgent.doStructuredCall()` trusts this unconditionally:

```java
if (model.supportsNativeStructuredOutput()) {
return doNativeStructuredCall(msgs, jsonSchema); // sends response_format → 400
}
```

## Reproduction

```java
ReActAgent agent = ReActAgent.builder()
.model(OpenAIChatModel.builder()
.baseUrl("https://api.deepseek.com")
.modelName("deepseek-chat")
.formatter(new DeepSeekFormatter())
.build())
.build();

agent.call(userMsg, OutputClass.class).block();
// DeepSeek: "This response_format type is unavailable now" (400)
```

## Affected Backends

All OpenAI-compatible APIs using `OpenAIChatModel` that don't support `response_format`: DeepSeek, vLLM, Ollama, SiliconFlow, TogetherAI (partial).

## Fix

Add automatic fallback in `doStructuredCall()`: when native path fails, retry with the fallback (generate_response tool) path.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.