workers-ai-provider: binding streams never honor abortSignal, so AI SDK timeout/abortSignal hang forever on stalled streams
- Dominant language
- TypeScript
- Stars
- 1.2k
- Forks
- 345
- Avg merge
- 13h 31m
- Merged PRs (30d)
- 1
Description
## Summary
When streaming through the Workers AI **binding** (`createWorkersAI({ binding: env.AI })`), the provider does not interrupt a pending read on the binding's `ReadableStream` when the request's `AbortSignal` fires. Since AI SDK v7 enforces `streamText({ timeout })` and `abortSignal` **solely** by aborting the signal it passes to `doStream` (there is no SDK-side read race), a stream that stalls without closing hangs the consumer forever — `timeout: { chunkMs, firstChunkMs, totalMs }` and caller-side `AbortController.abort()` are all silently ineffective.
This matters in practice because Workers AI streams do occasionally stall mid-generation without closing (we've observed this in production), which is exactly the situation timeouts exist for.
## Versions
- `workers-ai-provider` 4.0.0 (also reproduces on 3.3.1 with ai v6)
- `ai` 7.0.66
- Node 24 / workerd — reproduces in both
## Minimal repro
```js
import { streamText } from "ai";
import { createWorkersAI } from "workers-ai-provider";
const enc = new TextEncoder();
// Binding whose stream emits one chunk, then stalls (never closes) —
// mimics a stalled Workers AI stream.
const binding = {
run: async () =>
new ReadableStream({
start(c) {
c.enqueue(enc.encode(`data: ${JSON.stringify({ response: "Hello" })}\n\n`));
// deliberately no close(), no further chunks
},
}),
};
const model = createWorkersAI({ binding })("@cf/meta/llama-3.3-70b-instruct-fp8-fast");
const result = streamText({
model,
prompt: "hi",
timeout: { firstChunkMs: 500, chunkMs: 500, totalMs: 2000 },
});
for await (const t of result.textStream) console.log("token:", t);
console.log("done"); // never reached — process hangs (unsettled await)
```
Expected: the stream ends (SDK abort semantics) within ~500 ms and `onAbort` fires / result promises reject with `TimeoutError`.
Actual: hangs forever. Same behavior when using `abortSignal` with a manual `AbortController` instead of `timeout`.
## Root cause
`doStream` forwards the signal to the binding (`this.config.binding.run(model, inputs, { signal: options.abortSignal })`), but once the `ReadableStream` is returned, the pipeline in `getMappedStream` (`rawStream → SSEDecoder → TransformStream`) never observes the signal. A `reader.read()` that is pending when the signal fires is never rejected or cancelled, so the mapped stream never terminates. Fetch-based providers get abort-on-read for free from `fetch`; the binding path does not.
## Suggested fix
Race each read against the signal and error the mapped stream with `signal.reason` (which preserves the SDK's `TimeoutError`/`AbortError` semantics). We're running this in production as a `patch-package` patch:
```js
function raceAbort(stream, signal) {
if (!signal) return stream;
const reader = stream.getReader();
const aborted = new Promise((_, reject) => {
const fail = () => reject(signal.reason ?? new DOMException("Aborted", "AbortError"));
if (signal.aborted) fail();
else signal.addEventListener("abort", fail, { once: true });
});
aborted.catch(() => {});
return new ReadableStream({
async pull(controller) {
try {
const { done, value } = await Promise.race([reader.read(), aborted]);
if (done) controller.close();
else controller.enqueue(value);
} catch (e) {
reader.cancel(e).catch(() => {});
controller.error(e);
}
},
cancel(reason) {
return reader.cancel(reason);
},
});
}
```
applied at the `doStream` call site:
```js
getMappedStream(raceAbort(response, options.abortSignal), { ... })
```
With this in place, `streamText`'s `timeout` options and `abortSignal` behave per the SDK contract: the text stream ends cleanly, `onAbort` fires, and result promises reject with `TimeoutError`. Verified with unit tests against stalled fake streams and against live Workers AI (a 1 ms `firstChunkMs` terminates a real stream in ~200 ms).
Happy to turn this into a PR if useful.
Contributor guide
Research direction
Start at the binding path in doStream and follow the response into getMappedStream, focusing on how pending ReadableStream reads react to options.abortSignal. Reproduce the stalled fake stream from the issue, then add unit coverage for timeout and manual abort behavior; done means the mapped stream terminates and preserves signal.reason, including TimeoutError or AbortError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend-api-design, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 74/100