cloudflare / cloudflare/ai

[ai-gateway-provider] Dynamic Workers AI route fails through universal endpoint but succeeds through /compat

Open
#617 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
1.2k
Forks
345
Avg merge
13h 31m
Merged PRs (30d)
1

Description

## Summary

A documented `ai-gateway-provider` dynamic route targeting Workers AI fails when it is sent through the AI Gateway universal endpoint. The same model, prompt, and OpenAI-compatible request succeeds when sent directly to the gateway's `/compat/chat/completions` endpoint.

This also reproduces with a raw universal-endpoint request, without `ai-gateway-provider`, so the failure appears to be in the universal endpoint's dynamic-route -> Workers AI translation rather than the AI SDK request serializer.

## Environment

- `ai-gateway-provider@3.2.0`
- `ai@6.0.174`
- `@ai-sdk/openai-compatible@2.0.45`
- Node/Bun REST path using an authenticated AI Gateway
- Dynamic route target observed in response headers: `@cf/moonshotai/kimi-k2.7-code`

## Reproduction with `ai-gateway-provider`

```ts
import { generateText } from "ai";
import { createAiGateway } from "ai-gateway-provider";
import { createUnified } from "ai-gateway-provider/providers/unified";

const gateway = createAiGateway({
accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
gateway: process.env.CLOUDFLARE_AI_GATEWAY_ID!,
apiKey: process.env.CF_AIG_TOKEN!,
});
const unified = createUnified({ name: "cloudflare-ai-gateway" });

await generateText({
model: gateway(unified("dynamic/cloudflare")),
prompt: "Reply with exactly: OK",
maxOutputTokens: 64,
});
```

Result: HTTP 400.

Adding `parallel_tool_calls: false` through OpenAI-compatible provider options does not change the result.

## Minimal raw universal-endpoint reproduction

```bash
curl -X POST \
"https://gateway.ai.cloudflare.com/v1//" \
-H "cf-aig-authorization: Bearer " \
-H "content-type: application/json" \
--data '[{
"provider": "compat",
"endpoint": "chat/completions",
"headers": {},
"query": {
"model": "dynamic/cloudflare",
"messages": [{"role": "user", "content": "Reply exactly OK"}],
"max_tokens": 64,
"parallel_tool_calls": false
}
}]'
```

Response:

```json
{
"name": "AiError",
"internalCode": 5006,
"httpCode": 400,
"description": "Error: anyOf at '/' not met: required properties at '/audio' are 'voice,format', Cannot read properties of undefined (reading 'parallel_tool_calls'), oneOf at '/' not met, 0 matches"
}
```

Cloudflare request ID: `3fb00f66-9124-4421-b607-79465a53cc6f`

## Control request that succeeds

Sending the same `query` object directly to the documented compat endpoint succeeds:

```bash
curl -X POST \
"https://gateway.ai.cloudflare.com/v1///compat/chat/completions" \
-H "cf-aig-authorization: Bearer " \
-H "content-type: application/json" \
--data '{
"model": "dynamic/cloudflare",
"messages": [{"role": "user", "content": "Reply exactly OK"}],
"max_tokens": 64,
"parallel_tool_calls": false
}'
```

Result: HTTP 200, model `@cf/moonshotai/kimi-k2.7-code`, response text `OK`.

Cloudflare request ID: `62b2a517-554c-431b-8dad-79f68729ffee`

The direct compat path was also verified with AI SDK text generation, streaming, and forced tool calling.

## Expected behavior

The documented pattern below should work for a dynamic route whose selected model is a Workers AI model:

```ts
gateway(unified("dynamic/"))
```

The universal endpoint should either forward the OpenAI-compatible query to the selected Workers AI model or return the same successful result as `/compat/chat/completions`.

## Actual behavior

The universal endpoint returns HTTP 400 during request-shape validation/translation before inference, while the direct compat endpoint succeeds with the same query.

## Impact

Applications using `ai-gateway-provider` cannot use documented dynamic routes that resolve to this Workers AI model through the package's REST universal-endpoint transport. A working application-level workaround is to use `@ai-sdk/openai-compatible` directly against `/compat`, but that bypasses `ai-gateway-provider`'s universal-endpoint fallback behavior.

## Testing gap noticed

The package README documents `aigateway(unified("dynamic/customer-support"))`, but the current live E2E suite appears to cover provider/BYOK routing rather than a dynamic route resolving to Workers AI. A dynamic-route E2E regression case may help prevent this path from going unverified.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the universal endpoint's dynamic-route translation for the documented ai-gateway-provider pattern, then compare it with the working /compat/chat/completions path and the raw universal-endpoint reproduction. Review the README example and existing live E2E coverage. Done means a dynamic route resolving to Workers AI succeeds through the universal endpoint and has a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.