cloudflare / cloudflare/ai

Workers AI Responses delegate does not normalize binding errors carrying retryable: true

Open
#654 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.2k
Forks
345
Avg merge
13h 31m
Merged PRs (30d)
1

Description

## Summary

On a workers-ai-provider catalog delegate path using a custom OpenAI Responses wire plugin and the AI.run binding transport, a plain binding exception carrying retryable: true reaches the AI SDK unchanged. It is not normalized to an APICallError with isRetryable: true, so the SDK treats it as terminal despite maxRetries: 2.

The flag is still present on the error reported to onError; the problem is classification, not deletion of the property.

## Standalone reproduction

https://github.com/acchou/workers-ai-retryable-error-repro

Reproduced at commit e6a7c25. No credentials, Cloudflare deployment, or live model calls are required.

~~~sh
git clone https://github.com/acchou/workers-ai-retryable-error-repro.git
cd workers-ai-retryable-error-repro
npm ci --ignore-scripts --workspaces=false
npm run reproduce
~~~

The reproduction uses the real AI SDK, workers-ai-provider, and @ai-sdk/openai adapters. A custom wire plugin explicitly selects createOpenAI(...).responses(modelId). It configures transport: 'run' and resume: false for openai/gpt-5.

Only the binding is replaced: its first AI.run call throws the error shape observed in our application:

~~~js
Object.assign(new Error('Network connection lost.'), { retryable: true })
~~~

A second call would return a synthetic Responses SSE stream that the real OpenAI adapter can parse.

## Expected and actual

Expected: the explicitly retryable binding failure permits a second call within the SDK retry budget, and the request recovers.

Actual: the binding is called once. onError receives the original plain Error (retryable: true, not an APICallError), and result.text rejects with AI_NoOutputGeneratedError. npm run reproduce exits 1 deliberately to report this failed recovery assertion.

## Comparison fix and validation

~~~sh
npm run fixed
npm test
~~~

The comparison wrapper normalizes explicitly flagged, non-cancellation binding errors into APICallError({ isRetryable: true, cause: error, ... }). It adds no retry loop. With this wrapper, the real SDK calls the binding twice, about two seconds apart, and returns Recovered; the command exits 0.

All five checks pass: reproduction of the existing bug, successful recovery after normalization, unclassified errors remaining terminal, cancellation remaining terminal, and retry-budget exhaustion. A green test suite documents the known bug plus the comparison behavior; it does not mean upstream is fixed.

## Versions

- Node 24.13.1
- workers-ai-provider 4.0.0
- ai 7.0.58
- @ai-sdk/openai 4.0.15
- @ai-sdk/provider 4.0.7

The repository includes the exact dependency lockfile. These are the tested versions, not a claim that every later version is affected.

## Scope and related report

This isolates an error-classification gap before an individual provider stream is returned. It does not manufacture or explain a real Cloudflare network outage, test mid-stream reconnection, or replay previously executed tools. The wrapper illustrates the required translation, without assuming the best upstream patch location.

Related: #436 concerns HTTP 400 gateway responses suppressing retries. This reproduction instead concerns an exception from the binding with no HTTP response/status available.

Would normalization at the binding-backed run transport boundary be the appropriate fix for the delegated-provider path?

Contributor guide

Open the contributing guide

Research direction

Run the standalone reproduction with npm ci --ignore-scripts --workspaces=false and npm run reproduce, then compare its behavior with npm run fixed and npm test. Trace the workers-ai-provider delegated Responses path through the AI.run transport boundary; done means explicitly retryable binding errors are classified for SDK retries while unclassified errors and cancellations remain terminal.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.