`@cloudflare/tanstack-ai`: non-stream binding fetch discards OpenAI-shaped `binding.run` responses — structured output fails with "expected object, received string"
- Dominant language
- TypeScript
- Stars
- 1.2k
- Forks
- 345
- Avg merge
- 13h 31m
- Merged PRs (30d)
- 1
Description
## Summary
With `@cloudflare/tanstack-ai` 0.2.1, a native Workers AI model, and the binding transport, `chat({ outputSchema })` fails with a misleading validation error:
```
Error: Validation failed: Invalid input: expected object, received string
code: 'structured-output-validation-failed'
```
The model is not at fault — it returns a valid, schema-conformant object. The adapter's **non-stream response wrapper** discards that output, because `binding.run` returns an OpenAI-shaped `chat.completion` body and the wrapper only understands the native `{ response: ... }` shape. This is a regression against the equivalent `workers-ai-provider` + AI SDK setup, whose response extraction handles both shapes.
## Reproducer
```ts
import { chat } from "@tanstack/ai";
import { createWorkersAiChat } from "@cloudflare/tanstack-ai";
import { z } from "zod";
const adapter = createWorkersAiChat("@cf/meta/llama-4-scout-17b-16e-instruct", {
binding: env.AI,
gateway: env.CF_AIG_ID,
});
await chat({
adapter,
systemPrompts: [systemPrompt],
messages: [{ role: "user", content: userMessage }],
outputSchema: z.object({ id: z.string().nullable() }),
});
```
## Actual
```
Error: Validation failed: Invalid input: expected object, received string
code: 'structured-output-validation-failed'
at runAgenticStructuredOutput (@tanstack/ai/dist/esm/activities/chat/index.js:1671)
cause: StandardSchemaValidationError
at parseWithStandardSchema (@tanstack/ai/.../tools/schema-converter.js:157)
at TextEngine.runStructuredFinalization (@tanstack/ai/.../chat/index.js:1303)
```
## Root cause
`wrapWorkersAiResultAsOpenAI` (`packages/tanstack-ai/src/utils/create-fetcher.ts`), on the non-stream path of `createWorkersAiBindingFetch`:
```ts
const responseObj = typeof result === "object" && result !== null ? result : { response: String(result) };
const message = {
role: "assistant",
content: typeof responseObj.response === "string" ? responseObj.response : /* … */ "",
};
```
The wrapper only understands the native `{ response: ... }` shape. But `binding.run` for this model (messages + `response_format`) returns an **OpenAI-shaped `chat.completion` body** — `{"choices":[{"message":{"content":"{\"id\":\"A1\"}"}}], "object":"chat.completion", …}` (vLLM passthrough fields like `routed_experts` included). That body has no `.response` key, so:
1. `content` becomes `""` — the model's valid constrained output is discarded
2. in `structuredOutput`, `JSON.parse("")` throws → `catch { data = rawText }` → `data = ""`, a string
3. → `Validation failed: Invalid input: expected object, received string`
The failure is deterministic for any model/route whose run endpoint returns the OpenAI shape.
Notably, the **streaming** transformer in the same file already handles this exact case:
```ts
if (parsed.choices !== undefined) { isOpenAiFormat = true; /* pass through */ }
```
The non-stream wrapper just never got the equivalent. It's the mirror image of #543 (native shape reaching a consumer that expects OpenAI shape — here an OpenAI shape reaching a wrapper that expects native).
**Ruled out:** the `{ name, strict, schema }` envelope the adapter sends as `response_format.json_schema`. Tested live via the AI Gateway `workers-ai/run/` REST endpoint and via `binding.run` directly — both the envelope and a bare JSON Schema engage constrained decoding correctly on this model, and the sibling `workers-ai-provider` distinction from #559 is not the issue here.
## A second defect: the silent parse fallback
[`adapters/workers-ai.ts` → `structuredOutput`](https://github.com/cloudflare/ai/blob/main/packages/tanstack-ai/src/adapters/workers-ai.ts) does:
```ts
try {
data = JSON.parse(rawText);
} catch {
data = rawText; // ← a failed parse becomes the raw string, with no signal
}
```
This is what turned an empty-content bug into a confusing type error about the user's schema. Whatever happens upstream of it, a non-JSON reply should not silently flow onward as a `string` — surfacing `rawText` in the error would have made the root cause obvious from the first stack trace.
## Fix (verified locally)
Pass OpenAI-shaped bodies through untouched, mirroring the stream transformer (patched `dist` locally; `chat({ outputSchema })` then returns the validated object against the live model):
```ts
function wrapWorkersAiResultAsOpenAI(result, model, salvageContext) {
if (typeof result === "object" && result !== null && Array.isArray(result.choices)) {
return new Response(JSON.stringify(result), { headers: { "Content-Type": "application/json" } });
}
// …existing native-shape wrapping…
}
```
Happy to turn this into a PR if useful.
## Environment
- `@cloudflare/tanstack-ai`: 0.2.1
- `@tanstack/ai`: 0.42.0
- Model: `@cf/meta/llama-4-scout-17b-16e-instruct`
- Transport: direct binding + gateway id (`{ binding: env.AI, gateway: env.CF_AIG_ID }`)
- Runtime: Cloudflare Workers
## Related
- #543 — the same shape mismatch on the gateway-binding path, in the opposite direction
Contributor guide
Research direction
Read packages/tanstack-ai/src/utils/create-fetcher.ts, especially the non-stream path of createWorkersAiBindingFetch, and compare it with the streaming transformer’s OpenAI-shape handling. Run the provided chat({ outputSchema }) reproducer with the Workers AI binding; done means OpenAI-shaped binding.run responses preserve structured output, native responses still work, and the silent parse fallback in adapters/workers-ai.ts has been assessed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100