cloudflare / cloudflare/ai

ai-gateway-provider: AiGatewayChatLanguageModel hardcodes `supportedUrls = {}`, defeating provider URL passthrough

Open
#545 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.2k
Forks
345
Avg merge
13h 31m
Merged PRs (30d)
1

Description

## Summary

`AiGatewayChatLanguageModel` in `ai-gateway-provider@3.1.3` ([dist/index.mjs:143](https://github.com/cloudflare/ai/blob/main/packages/ai-gateway-provider/src/index.ts)) hardcodes `supportedUrls = {}` instead of forwarding the underlying wrapped provider's `supportedUrls`. This silently disables URL passthrough for **every provider wrapped by `createAiGateway`**, causing AI SDK to download every file URL and inline it as base64 in the request body.

For a wrapped Anthropic model serving a real PDF (4.4 MiB), this means:

- AI SDK sees `supportedUrls === {}` from the gateway wrapper
- `isUrlSupported({ mediaType: 'application/pdf', supportedUrls: {} })` → `false`
- AI SDK downloads the PDF via `download()`
- `convertPartToLanguageModelPart` replaces URL with bytes
- Anthropic provider falls into the `base64` branch (~5.9 MiB after base64)
- Final Anthropic request body is ~6 MB inline data per request

The same PDF, when sent direct to Anthropic without the gateway wrapper, goes through as `source: { type: 'url', url: '...' }` — Anthropic fetches it server-side, request body stays a few KB.

## Reproducer

```ts
import { createAnthropic } from 'ai-gateway-provider/providers/anthropic'
import { createAiGateway } from 'ai-gateway-provider'

const anthropic = createAnthropic({ apiKey: 'sk-ant-...' })
const gw = createAiGateway({ accountId: '...', gateway: '...', apiKey: '...' })

const direct = anthropic('claude-sonnet-4-6')
const wrapped = gw(direct)

console.log('direct:', await direct.supportedUrls)
// { 'image/*': [/^https?:\/\/.*$/], 'application/pdf': [/^https?:\/\/.*$/] }

console.log('wrapped:', await wrapped.supportedUrls)
// {} ← bug
```

## Real-world impact

We hit this in production with two symptoms before tracing both to this bug:

1. **PDF analysis triggered SessionDO (Cloudflare Durable Object) OOM**: 128 MB isolate cap exceeded while AI SDK constructed the 6 MB request body around a 4.4 MiB PDF. Affected ~7 specific agents over a week, each in heavy use (~33 MiB VFS, dozens of accumulated sessions).
2. **Large image uploads hit Anthropic's 5 MiB base64 limit**: a 4 MB JPEG that should have been URL-passed-through was downloaded by AI SDK, base64'd to 5.6 MB, rejected by Anthropic with `messages.N.content.M.image.source.base64: image exceeds 5 MB maximum`.

Both are downstream effects of `supportedUrls = {}`.

## Expected behavior

`AiGatewayChatLanguageModel.supportedUrls` should forward the wrapped provider's `supportedUrls`, the same way `modelId` and `provider` already do via getters.

## Existing fix in PR #409

The full rewrite in #409 already implements this correctly:

```ts
// packages/ai-gateway-provider/src/index.ts (PR #409 HEAD)
get supportedUrls() {
return this.innerModel.supportedUrls; // single-model wrapper
}

get supportedUrls() {
return this.models[0]!.supportedUrls; // fallback wrapper
}
```

PR #409 has been open since 2026-03-22 with no reviews. It's a large rewrite that introduces a new API surface with backward-compat shims — sound work, but in the meantime users wrapped by `createAiGateway` are paying real costs (network egress, billable token recompute via cache misses, isolate memory pressure, request latency).

## Suggested resolution

Two reasonable options:

**a) Minimal targeted fix on the current major (3.x):** add a getter to the existing `AiGatewayChatLanguageModel` class in `src/index.ts`:

```diff
var AiGatewayChatLanguageModel = class {
get modelId() { ... }
get provider() { ... }
+ get supportedUrls() {
+ return this.models[0]?.supportedUrls ?? {};
+ }
constructor(models, config) {
_defineProperty(this, "specificationVersion", "v3");
_defineProperty(this, "defaultObjectGenerationMode", "json");
- _defineProperty(this, "supportedUrls", {});
_defineProperty(this, "models", void 0);
...
}
}
```

This is what we're currently shipping as a [pnpm patch](https://pnpm.io/cli/patch) — verified to restore URL passthrough end-to-end (PDF source type goes from `base64` to `url`; request body shrinks from 6 MB to a few KB). Three-line diff, zero behavior change for callers that don't supply file URLs.

**b) Land PR #409**, which already contains the equivalent fix and additional improvements.

Happy to send a PR with just (a) if it would help unblock affected users while #409 is in flight — just let me know.

## Environment

- `ai-gateway-provider@3.1.3` (latest)
- `@ai-sdk/anthropic@3.0.76`
- `ai@6.0.177`
- Cloudflare Workers + Durable Objects runtime

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.