cloudwego / cloudwego/eino-ext

[openai] Structured-output knobs not wired through: WithResponseFormat (documented but missing) and FunctionDefinition.Strict (dropped during marshal)

Open
#856 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
811
Forks
368
Avg merge
16h 22m
Merged PRs (30d)
13

Description

## Summary

Two provider-specific structured-output knobs are surfaced in `libs/acl/openai`'s README but cannot actually be used at the public API surface:

1. **`openai.WithResponseFormat(format)`** is documented as a per-call option but **does not exist in the source** — neither in `libs/acl/openai` nor in `components/model/openai`.
2. **`FunctionDefinition.Strict`** is supported by the upstream Go SDK (`meguminnnnnnnnn/go-openai`) but eino-ext's tool-marshal path **drops it on the floor**, and `schema.ToolInfo` has no input field for it either.

Both gaps force users into untyped or brittle workarounds when they want to enable DeepSeek `json_object` mode or OpenAI Structured Outputs `strict: true` per call.

## Evidence

### (a) `WithResponseFormat` is documented but missing

[`libs/acl/openai/README.md`](https://github.com/cloudwego/eino-ext/blob/main/libs/acl/openai/README.md#json-schema-response) shows:

```go
resp, err := client.Generate(ctx, messages,
model.WithGenOption(openai.WithResponseFormat(jsonSchema)),
)
```

and under "Available call options":

> // Response format
> openai.WithResponseFormat(format)

But:

```console
$ grep -rn "func WithResponseFormat" libs/acl/openai/ components/model/openai/
$ # (no matches in v0.1.17 nor in main)
```

`libs/acl/openai/option.go` only defines: `WithExtraFields`, `WithReasoningEffort`, `WithRequestPayloadModifier`, `WithResponseMessageModifier`, `WithResponseChunkMessageModifier`, `WithRequestBodyModifier` (deprecated), `WithExtraHeader`, `WithMaxCompletionTokens`. No `WithResponseFormat`.

`components/model/openai/option.go` correspondingly does not re-export it.

The only way to set `response_format` today is via whole-model `Config.ResponseFormat` at `NewChatModel` time — which means a single ChatModel cannot mix calls that need `json_object` (e.g. a structured-extraction helper) with calls that must stay free-form.

For completeness: `model.WithGenOption` referenced in that README example is also not exported by `cloudwego/eino@v0.9.1/components/model/option.go`.

### (b) `FunctionDefinition.Strict` is dropped during tool marshal

Upstream SDK already supports it — `meguminnnnnnnnn/go-openai@v0.1.2/chat.go:454-461`:

```go
type FunctionDefinition struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Strict bool `json:"strict,omitempty"`
Parameters any `json:"parameters"`
}
```

eino-ext's marshal site silently omits it — `libs/acl/openai/chat_model.go:655-662`:

```go
req.Tools[i] = openai.Tool{
Type: openai.ToolTypeFunction,
Function: &openai.FunctionDefinition{
Name: t.Function.Name,
Description: t.Function.Description,
Parameters: t.Function.Parameters,
// Strict is missing here.
},
}
```

And `eino.schema.ToolInfo.Function` exposes no `Strict` field, so there is no caller input path either.

## Why this matters

A common pattern is a single shared `ChatModel` that serves two kinds of calls:

- A **structured-extraction call** that needs `response_format: {"type":"json_object"}` (or `json_schema`) so the server guarantees a syntactically valid JSON body. With DeepSeek specifically, without this directive the server occasionally emits unescaped control bytes inside string values that `sonic`'s strict default rejects with `Syntax error at index N: invalid char`.
- A **free-form text call** on the same model that must NOT have `response_format` set.

There is no clean per-call switch today. The workarounds:

- `WithExtraFields(map[string]any{"response_format": map[string]any{"type": "json_object"}})` — works, but untyped; defeats the purpose of the typed `ChatCompletionResponseFormat` already in the package.
- Build two separate `ChatModel` instances with different `Config.ResponseFormat` — duplicates HTTP client / base URL / retry config; awkward when the shared model is the natural design.

Same story for `Strict` on tool functions: anyone wanting OpenAI Structured Outputs with `strict: true` or DeepSeek's tool-call strict mode has to reach for `WithRequestPayloadModifier` and patch the serialized JSON post-hoc.

References:
- DeepSeek JSON Output: https://api-docs.deepseek.com/zh-cn/guides/json_mode
- DeepSeek Tool Calls (strict): https://api-docs.deepseek.com/zh-cn/guides/function_calling
- OpenAI Structured Outputs: https://platform.openai.com/docs/guides/structured-outputs

## Proposed direction

### (a) — small, self-contained

1. Add `ResponseFormat *ChatCompletionResponseFormat` to `openaiOptions` in `libs/acl/openai/option.go`.
2. Add `WithResponseFormat(format *ChatCompletionResponseFormat) model.Option`.
3. In `genRequest` in `chat_model.go`, prefer `specOptions.ResponseFormat` over `c.config.ResponseFormat`.
4. Re-export `WithResponseFormat` from `components/model/openai/option.go` to match the existing pattern of the other `Withxxx` functions there.

### (b) — needs maintainer design call

The caller-facing question is whether to:
- add a `Strict bool` field to `schema.ToolInfo.Function` upstream in `cloudwego/eino`, or
- accept it via an openai-only option (e.g. a per-tool `Strict` overlay), or
- read it from `schema.ToolInfo.Extra`.

Either way, the marshal-site fix in `libs/acl/openai/chat_model.go:657` is one line:

```go
Function: &openai.FunctionDefinition{
Name: t.Function.Name,
Description: t.Function.Description,
Parameters: t.Function.Parameters,
Strict: /* sourced from chosen input path */,
},
```

I'm happy to send the PR for (a) once the team confirms the API shape. (b) is left as a discussion item since the input-path choice is yours to make.

---

Filed by a downstream user; happy to iterate. Thanks for the framework!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.