[ai-gateway-provider] Dynamic Workers AI route fails through universal endpoint but succeeds through /compat
- Dominant language
- TypeScript
- Stars
- 1.2k
- Forks
- 345
- Avg merge
- 13h 31m
- Merged PRs (30d)
- 1
Description
## Summary
A documented `ai-gateway-provider` dynamic route targeting Workers AI fails when it is sent through the AI Gateway universal endpoint. The same model, prompt, and OpenAI-compatible request succeeds when sent directly to the gateway's `/compat/chat/completions` endpoint.
This also reproduces with a raw universal-endpoint request, without `ai-gateway-provider`, so the failure appears to be in the universal endpoint's dynamic-route -> Workers AI translation rather than the AI SDK request serializer.
## Environment
- `ai-gateway-provider@3.2.0`
- `ai@6.0.174`
- `@ai-sdk/openai-compatible@2.0.45`
- Node/Bun REST path using an authenticated AI Gateway
- Dynamic route target observed in response headers: `@cf/moonshotai/kimi-k2.7-code`
## Reproduction with `ai-gateway-provider`
```ts
import { generateText } from "ai";
import { createAiGateway } from "ai-gateway-provider";
import { createUnified } from "ai-gateway-provider/providers/unified";
const gateway = createAiGateway({
accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
gateway: process.env.CLOUDFLARE_AI_GATEWAY_ID!,
apiKey: process.env.CF_AIG_TOKEN!,
});
const unified = createUnified({ name: "cloudflare-ai-gateway" });
await generateText({
model: gateway(unified("dynamic/cloudflare")),
prompt: "Reply with exactly: OK",
maxOutputTokens: 64,
});
```
Result: HTTP 400.
Adding `parallel_tool_calls: false` through OpenAI-compatible provider options does not change the result.
## Minimal raw universal-endpoint reproduction
```bash
curl -X POST \
"https://gateway.ai.cloudflare.com/v1//" \
-H "cf-aig-authorization: Bearer " \
-H "content-type: application/json" \
--data '[{
"provider": "compat",
"endpoint": "chat/completions",
"headers": {},
"query": {
"model": "dynamic/cloudflare",
"messages": [{"role": "user", "content": "Reply exactly OK"}],
"max_tokens": 64,
"parallel_tool_calls": false
}
}]'
```
Response:
```json
{
"name": "AiError",
"internalCode": 5006,
"httpCode": 400,
"description": "Error: anyOf at '/' not met: required properties at '/audio' are 'voice,format', Cannot read properties of undefined (reading 'parallel_tool_calls'), oneOf at '/' not met, 0 matches"
}
```
Cloudflare request ID: `3fb00f66-9124-4421-b607-79465a53cc6f`
## Control request that succeeds
Sending the same `query` object directly to the documented compat endpoint succeeds:
```bash
curl -X POST \
"https://gateway.ai.cloudflare.com/v1///compat/chat/completions" \
-H "cf-aig-authorization: Bearer " \
-H "content-type: application/json" \
--data '{
"model": "dynamic/cloudflare",
"messages": [{"role": "user", "content": "Reply exactly OK"}],
"max_tokens": 64,
"parallel_tool_calls": false
}'
```
Result: HTTP 200, model `@cf/moonshotai/kimi-k2.7-code`, response text `OK`.
Cloudflare request ID: `62b2a517-554c-431b-8dad-79f68729ffee`
The direct compat path was also verified with AI SDK text generation, streaming, and forced tool calling.
## Expected behavior
The documented pattern below should work for a dynamic route whose selected model is a Workers AI model:
```ts
gateway(unified("dynamic/"))
```
The universal endpoint should either forward the OpenAI-compatible query to the selected Workers AI model or return the same successful result as `/compat/chat/completions`.
## Actual behavior
The universal endpoint returns HTTP 400 during request-shape validation/translation before inference, while the direct compat endpoint succeeds with the same query.
## Impact
Applications using `ai-gateway-provider` cannot use documented dynamic routes that resolve to this Workers AI model through the package's REST universal-endpoint transport. A working application-level workaround is to use `@ai-sdk/openai-compatible` directly against `/compat`, but that bypasses `ai-gateway-provider`'s universal-endpoint fallback behavior.
## Testing gap noticed
The package README documents `aigateway(unified("dynamic/customer-support"))`, but the current live E2E suite appears to cover provider/BYOK routing rather than a dynamic route resolving to Workers AI. A dynamic-route E2E regression case may help prevent this path from going unverified.
Contributor guide
Research direction
Start by tracing the universal endpoint's dynamic-route translation for the documented ai-gateway-provider pattern, then compare it with the working /compat/chat/completions path and the raw universal-endpoint reproduction. Review the README example and existing live E2E coverage. Done means a dynamic route resolving to Workers AI succeeds through the universal endpoint and has a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100