Feature: official generation middleware for stream smoothing

Problem

Some models (notably Gemini) tend to stream their output in large, bursty chunks rather than a steady token-by-token flow. When this output is piped straight into a client "typewriter" UI, the result feels janky: long pauses followed by big blocks of text appearing all at once, instead of smooth, continuous typing.

Today every app that wants a polished streaming UX has to solve this themselves, either on the client (buffering + re-pacing in the browser) or with bespoke server middleware. This is common enough that it belongs in the framework.

Proposal

Add an official generation/model middleware (e.g. smoothStream) that buffers incoming streamed text and re-emits it as smaller pieces paced at a steady cadence, so text appears to flow smoothly regardless of how the underlying model batches its tokens. This mirrors prior art such as the Vercel AI SDK's smoothStream.

Suggested API

const { stream } = ai.generateStream({
  model: gemini20Flash,
  prompt: 'Tell me a story.',
  use: [smoothStream({ delayMs: 15, chunking: 'word' })],
});

Options to consider:

• delayMs - target delay between emitted pieces (e.g. default ~20ms).
• chunking - 'word' (default), 'char', or a custom RegExp for boundaries like CJK.
• chunkSize - number of granular units to bundle per emitted chunk.

Behavior / requirements

• Implemented as a ModelMiddlewareWithOptions so it can intercept the streaming onChunk callback.
• Only re-split plain incremental text chunks. Chunks carrying media, tool requests, reasoning, custom data, or the aggregated flag should be flushed in order and passed through untouched.
• No-op when the request is not streaming.
• Must flush all buffered content before the generate call resolves, so no streamed text is dropped.
• Preserve text exactly (lossless reconstruction) and preserve chunk role/index.

Possible enhancement

A fixed per-piece delay is simple and effective, but can add latency if a model dumps a very large buffer at once. A follow-up could support adaptive pacing (spread the currently buffered text over an estimated window, or cap buffer/latency).

Prototype

I have a working prototype of this middleware (with tests) that re-splits bursty chunks into evenly-paced word/char pieces. Happy to open a PR if there's interest in the design.

Start by locating ModelMiddlewareWithOptions, the generateStream entry point, and the streaming onChunk callback; review how existing middleware handles non-text chunks and completion. Use the prototype and its tests as reference if available. Done means buffered plain text is re-emitted with the requested pacing and chunking, special chunks pass through in order, streaming is lossless, and all buffered content flushes before generation resolves.