[tanstack-ai] Support resumable run path for third-party provider adapters
- Dominant language
- TypeScript
- Stars
- 1.2k
- Forks
- 345
- Avg merge
- 13h 31m
- Merged PRs (30d)
- 1
Description
## Context
In `@cloudflare/tanstack-ai`, the dedicated third-party provider adapters (`createOpenAiChat`, `createGrokChat`, `createGeminiChat`, `createAnthropicChat`, `createOpenRouterChat`) route through the AI Gateway **universal endpoint** (`env.AI.gateway(id).run({ provider, endpoint, headers, query })` via `createGatewayFetch`). That path does not surface a `cf-aig-run-id`, so it cannot use the resumable-stream engine.
Only the `workers-ai` adapter's binding shim (`createWorkersAiBindingFetch`) uses the resumable **run path** (`env.AI.run(model, inputs, { gateway, returnRawResponse })`), and only for `@cf/*` models and (now) `"/"` catalog slugs.
By contrast, `workers-ai-provider`'s gateway delegate can run third-party unified-billing catalog models on the resumable run path.
## Ask
Give the third-party provider adapters an option to dispatch through the unified-billing **run path** (`env.AI.run`) where the model is on Cloudflare's unified run catalog, so they can:
- emit `cf-aig-run-id` and support resumable streaming (`createResumableStream`), and
- get unified billing on the run path consistently with `workers-ai-provider`.
## Notes / considerations
- Not a bug — this is a feature gap. The universal endpoint already provides unified billing (key omitted) and BYOK (key present); this is specifically about **resume**.
- Each adapter is built on its provider SDK (`@tanstack/ai-openai`, etc.), so the run-path fetch would need to be wired per adapter (or via a shared helper) rather than the current `createGatewayFetch`.
- Run-catalog membership should reuse the shared `@cloudflare/gateway-core` registry (`runCatalog`/`billing`) rather than duplicating classification.
- Only providers actually on the unified run catalog qualify (empirically: openai, anthropic, google, xai, groq, deepseek, alibaba, minimax); the rest stay on the gateway/BYOK path.
Follow-up from the #596 work.
Contributor guide
Research direction
Start by reading the named third-party adapters and the createGatewayFetch and createWorkersAiBindingFetch entry points. Trace how createResumableStream consumes cf-aig-run-id, then inspect the shared @cloudflare/gateway-core runCatalog/billing registry. Done means qualifying catalog models use env.AI.run with resumable headers while unsupported providers retain the existing gateway/BYOK path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100