cloudflare / cloudflare/ai

[tanstack-ai] Support resumable run path for third-party provider adapters

Open
#598 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
1.2k
Forks
345
Avg merge
13h 31m
Merged PRs (30d)
1

Description

## Context

In `@cloudflare/tanstack-ai`, the dedicated third-party provider adapters (`createOpenAiChat`, `createGrokChat`, `createGeminiChat`, `createAnthropicChat`, `createOpenRouterChat`) route through the AI Gateway **universal endpoint** (`env.AI.gateway(id).run({ provider, endpoint, headers, query })` via `createGatewayFetch`). That path does not surface a `cf-aig-run-id`, so it cannot use the resumable-stream engine.

Only the `workers-ai` adapter's binding shim (`createWorkersAiBindingFetch`) uses the resumable **run path** (`env.AI.run(model, inputs, { gateway, returnRawResponse })`), and only for `@cf/*` models and (now) `"/"` catalog slugs.

By contrast, `workers-ai-provider`'s gateway delegate can run third-party unified-billing catalog models on the resumable run path.

## Ask

Give the third-party provider adapters an option to dispatch through the unified-billing **run path** (`env.AI.run`) where the model is on Cloudflare's unified run catalog, so they can:

- emit `cf-aig-run-id` and support resumable streaming (`createResumableStream`), and
- get unified billing on the run path consistently with `workers-ai-provider`.

## Notes / considerations

- Not a bug — this is a feature gap. The universal endpoint already provides unified billing (key omitted) and BYOK (key present); this is specifically about **resume**.
- Each adapter is built on its provider SDK (`@tanstack/ai-openai`, etc.), so the run-path fetch would need to be wired per adapter (or via a shared helper) rather than the current `createGatewayFetch`.
- Run-catalog membership should reuse the shared `@cloudflare/gateway-core` registry (`runCatalog`/`billing`) rather than duplicating classification.
- Only providers actually on the unified run catalog qualify (empirically: openai, anthropic, google, xai, groq, deepseek, alibaba, minimax); the rest stay on the gateway/BYOK path.

Follow-up from the #596 work.

Contributor guide

Open the contributing guide

Research direction

Start by reading the named third-party adapters and the createGatewayFetch and createWorkersAiBindingFetch entry points. Trace how createResumableStream consumes cf-aig-run-id, then inspect the shared @cloudflare/gateway-core runCatalog/billing registry. Done means qualifying catalog models use env.AI.run with resumable headers while unsupported providers retain the existing gateway/BYOK path.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.