cloudflare / cloudflare/cloudflare-os

`apiUrl` can't reach an authenticated AI Gateway: the direct path has no way to send `cf-aig-authorization`

Open
#253 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
9.9k
Forks
1.2k
Avg merge
1d 20h
Merged PRs (30d)
107

Description

`AiModelConfig.apiUrl` documents itself as the way to point a model at a proxy:

https://github.com/cloudflare/cloudflare-os/blob/main/packages/workshop-shared/src/api.ts#L1160-L1165

> URL of the API. If not specified, use the default for the provider. Overriding the URL is
> useful in order to use AI proxy products like Cloudflare's AI gateway, or even to use an
> alternative provider that provides a compatible API.

But an AI Gateway with **Authenticated Gateway** turned on rejects a request that carries no `cf-aig-authorization` header, and the direct path has no way to send one. In `ai-models.ts` that header is only ever constructed in `getModelViaUserGateway()` and `getModelViaGateway()` — both reached from a connected Cloudflare account or the `CF_AI_GATEWAY_*` deployment env, never from the model config:

```
$ rg -n 'cf-aig-authorization' packages/workshop-backend/src/ai-models.ts
400: "cf-aig-authorization": `Bearer ${userGateway.apiKey}`, # getModelViaUserGateway
427: "cf-aig-authorization": `Bearer ${gwConfig.apiToken}`, # getModelViaGateway
```

`getModelDirect()` builds `headers` only for the keyless-Ollama `{Authorization: null}` case. `AiModelConfig` is `{provider, model, apiToken, accountId?, apiUrl?}` — there is no field that can carry it.

So the documented use case works only against an *unauthenticated* gateway. The same gap applies to any header-authenticated proxy (LiteLLM, Portkey, Helicone) and to private inference endpoints that route on a header such as `X-Tenant-Id`.

To be upfront: the fix is an added field, so this is capability-shaped rather than a pure bug, and it is over the automatic PR size limit — I'm filing it rather than sending a patch. Discussion #29 covers the adjacent request-shape problem, but nothing there or anywhere else in the tracker mentions headers (`custom header`, `extraHeaders`, `defaultHeaders` return zero results).

### What would fix it

`headers?: Record` on `AiModelConfig`, merged in `getModelDirect()`. pi already supports it — `ProviderHeaders` exists and all four API impls merge `options.headers` last so caller values beat the SDK's own; it is simply never populated from user config.

Two things that matter if anyone picks this up:

- **Merge order is load-bearing.** The config's headers have to merge *after* the ones the provider client sends. The keyless Ollama path sends `Authorization: null` to strip the SDK's bearer token, so if config headers merge first, a user-supplied `Authorization` is deleted and the endpoint gets no credentials at all.
- **Direct path only**, matching how `apiToken`/`apiUrl` are already ignored under AI Gateway mode — otherwise a config header could clobber `cf-aig-authorization` on the gateway paths.

I have this working on a fork if it's useful as a reference — about 40 lines of non-test change, plus unit tests and an end-to-end check against a local endpoint with a recording proxy in front of it, asserting on what actually reached the wire: https://github.com/ishibashi-futos/cloudflare-os/pull/1

Happy to close this if you'd rather fold it into the custom-provider work in #29.

Contributor guide

Open the contributing guide

Research direction

Start with AiModelConfig in packages/workshop-shared/src/api.ts and getModelDirect() in packages/workshop-backend/src/ai-models.ts, then inspect pi's ProviderHeaders handling and the existing unit tests. Verify the direct path against a local recording proxy, including authenticated headers and the keyless Ollama case, while confirming gateway paths remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
authentication, backend-api-design
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.