cloudflare / cloudflare/ai

AI Gateway: Support custom base_url for native provider adapters

Open
#544 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.2k
Forks
345
Avg merge
13h 31m
Merged PRs (30d)
1

Description

## Feature request

AI Gateway currently supports native providers such as Anthropic, Google AI Studio, OpenAI, Azure OpenAI, and others. It also supports BYOK / Store Keys for storing provider API keys in Cloudflare, and it supports Custom Providers with a custom `base_url`.

However, Custom Providers do not appear to support reusing native provider adapters with a user-defined `base_url`.

We would like AI Gateway to support “provider-compatible custom endpoints”, for example:

```yaml
provider_type: anthropic-compatible
base_url: https://custom-anthropic-compatible-provider.example.com
auth: stored BYOK key
adapter: reuse Anthropic adapter
expose_via: /compat/chat/completions
```

and:

```yaml
provider_type: google-ai-studio-compatible
base_url: https://custom-gemini-compatible-provider.example.com
auth: stored BYOK key
adapter: reuse Google AI Studio adapter
path_template:
generate: /v1beta/models/{model}:generateContent
stream: /v1beta/models/{model}:streamGenerateContent
expose_via: /compat/chat/completions
```

## Use case

Many upstream vendors expose Anthropic-compatible or Google-AI-Studio-compatible APIs under their own domains. They are protocol-compatible with the official provider APIs, but their base URLs are different.

Today, AI Gateway users have two separate capabilities:

1. Native providers with protocol adapters and BYOK / Store Keys.
2. Custom Providers with custom `base_url`.

But there does not seem to be a way to combine them:

- use the Anthropic / Google AI Studio native adapter;
- override the official provider `base_url`;
- store the upstream API key in AI Gateway BYOK / Provider Keys;
- continue to call the provider through `/compat/chat/completions`;
- preserve streaming, usage parsing, logging, retries, fallback, rate limiting, and cost analytics.

Without this capability, users need to build and operate their own adapter Workers only to change the provider base URL.

## Expected behavior

AI Gateway should allow creating a custom endpoint that reuses an existing native provider adapter:

- adapter type: Anthropic / Google AI Studio / OpenAI-compatible / Azure-compatible;
- custom `base_url`;
- optional path template;
- stored API key via BYOK / Provider Keys;
- model prefix such as `custom-anthropic/{model}` or `custom-google-ai-studio/{model}`;
- support through `/compat/chat/completions`.

This is different from the current generic Custom Provider feature. The request is not merely to proxy arbitrary HTTPS APIs. The request is to reuse existing native provider adapters with a user-defined `base_url`.

## Why this matters

This would make AI Gateway much more useful as a universal AI routing layer. It would reduce the need for users to build separate adapter Workers, and it would allow AI Gateway to manage more upstream providers securely while preserving its existing observability, caching, rate limiting, retries, fallback, and cost analytics features.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.