cloudflare / cloudflare/agents

Think: no hook to set AI Gateway id or `cf-aig-metadata` per turn

Open
#2,262 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

## Summary

`@cloudflare/think@0.17.0` resolves a string `getModel()` through the bundled `workers-ai-provider` catalog. That path always uses AI Gateway id `default` and has no public way to set `cf-aig-metadata`. The only override that can change either is `resolveModel`, so any app that bills or traces on `{ role, siteId }` has to re-implement model resolution.

`TurnConfig` exposes `headers` and `providerOptions` per turn, but not a gateway id, and `cf-aig-metadata` on the catalog path is the provider's `{ metadata }` argument, not an HTTP header.

## Environment

- `@cloudflare/think@0.17.0`
- `Think.getModel()` returns `"/"` (e.g. `openai/gpt-5.6-terra`)
- Workers AI binding + AI Gateway

## What the types allow

`Think.resolveModel` (`dist/index-B7zEkBBM.d.ts`):

```ts
resolveModel(model?: ThinkModel): LanguageModel;
```

`getModel()` documents that a `"/"` slug is routed through AI Gateway. There is no `getGateway()`, no `gateway` field on `TurnConfig`, and no typed metadata slot.

## Reproduction

1. Subclass `Think`, return `"openai/gpt-5.6-terra"` from `getModel()`, leave `resolveModel` as the default.
2. Run a turn that calls tools with `reasoning_effort`.
3. Observe:
- the request hits AI Gateway id `default`
- no `cf-aig-metadata`
- OpenAI Chat Completions 400s on gpt-5.6-terra with tools + `reasoning_effort` (the catalog plugin uses `.chat()`)

## Workaround

Override `resolveModel` and construct the providers yourself. Ours (`apps/agent/src/think.ts`):

```ts
override resolveModel(model = this.getModel()): LanguageModel {
if (typeof model !== "string") return model;
const metadata = { role: this.role, siteId: this.siteId };
if (model.startsWith("openai/")) {
this.#openai ??= createGatewayProvider(createOpenAI, {
binding: this.getAIBinding(),
gateway: { id: "clutch" },
extraHeaders: { "cf-aig-metadata": JSON.stringify(metadata) },
});
return this.#openai(model.slice("openai/".length));
}
this.#catalog ??= createWorkersAI({
binding: this.getAIBinding(),
gateway: { id: "clutch" },
providers: [openai, anthropic],
});
return this.#catalog(model, { metadata });
}
```

`openai/` goes through `createGatewayProvider(createOpenAI, …)` so it stays on the Responses API. Everything else goes through the catalog with `{ metadata }`. Both pin gateway id `clutch`.

## Suggested fix

A hook (or `TurnConfig` fields) to set gateway `{ id }` and `cf-aig-metadata` without replacing `resolveModel`. Default resolution can keep `default` and empty metadata.

Contributor guide

Open the contributing guide

Research direction

Start with Think.resolveModel and TurnConfig in dist/index-B7zEkBBM.d.ts, then compare the workaround in apps/agent/src/think.ts with the default catalog path. Reproduce the gateway and metadata behavior described for a tool-calling turn, and verify that the chosen API lets callers set both values while preserving the default gateway and empty metadata behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.