cloudflare / cloudflare/agents
Think: no hook to set AI Gateway id or `cf-aig-metadata` per turn
- Dominant language
- TypeScript
- Stars
- 5.6k
- Forks
- 711
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 53
Description
## Summary
`@cloudflare/think@0.17.0` resolves a string `getModel()` through the bundled `workers-ai-provider` catalog. That path always uses AI Gateway id `default` and has no public way to set `cf-aig-metadata`. The only override that can change either is `resolveModel`, so any app that bills or traces on `{ role, siteId }` has to re-implement model resolution.
`TurnConfig` exposes `headers` and `providerOptions` per turn, but not a gateway id, and `cf-aig-metadata` on the catalog path is the provider's `{ metadata }` argument, not an HTTP header.
## Environment
- `@cloudflare/think@0.17.0`
- `Think.getModel()` returns `"/"` (e.g. `openai/gpt-5.6-terra`)
- Workers AI binding + AI Gateway
## What the types allow
`Think.resolveModel` (`dist/index-B7zEkBBM.d.ts`):
```ts
resolveModel(model?: ThinkModel): LanguageModel;
```
`getModel()` documents that a `"/"` slug is routed through AI Gateway. There is no `getGateway()`, no `gateway` field on `TurnConfig`, and no typed metadata slot.
## Reproduction
1. Subclass `Think`, return `"openai/gpt-5.6-terra"` from `getModel()`, leave `resolveModel` as the default.
2. Run a turn that calls tools with `reasoning_effort`.
3. Observe:
- the request hits AI Gateway id `default`
- no `cf-aig-metadata`
- OpenAI Chat Completions 400s on gpt-5.6-terra with tools + `reasoning_effort` (the catalog plugin uses `.chat()`)
## Workaround
Override `resolveModel` and construct the providers yourself. Ours (`apps/agent/src/think.ts`):
```ts
override resolveModel(model = this.getModel()): LanguageModel {
if (typeof model !== "string") return model;
const metadata = { role: this.role, siteId: this.siteId };
if (model.startsWith("openai/")) {
this.#openai ??= createGatewayProvider(createOpenAI, {
binding: this.getAIBinding(),
gateway: { id: "clutch" },
extraHeaders: { "cf-aig-metadata": JSON.stringify(metadata) },
});
return this.#openai(model.slice("openai/".length));
}
this.#catalog ??= createWorkersAI({
binding: this.getAIBinding(),
gateway: { id: "clutch" },
providers: [openai, anthropic],
});
return this.#catalog(model, { metadata });
}
```
`openai/` goes through `createGatewayProvider(createOpenAI, …)` so it stays on the Responses API. Everything else goes through the catalog with `{ metadata }`. Both pin gateway id `clutch`.
## Suggested fix
A hook (or `TurnConfig` fields) to set gateway `{ id }` and `cf-aig-metadata` without replacing `resolveModel`. Default resolution can keep `default` and empty metadata.
Contributor guide
Research direction
Start with Think.resolveModel and TurnConfig in dist/index-B7zEkBBM.d.ts, then compare the workaround in apps/agent/src/think.ts with the default catalog path. Reproduce the gateway and metadata behavior described for a tool-calling turn, and verify that the chosen API lets callers set both values while preserving the default gateway and empty metadata behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100