Stream writes carry no idempotency key, so a retried step re-appends its chunks

streams.write() is a bare append — no chunk index, no dedupe key — so when a step is retried, every chunk it already wrote gets written again and readers see the duplicates. The only mitigation available to users today is "keep your chunks idempotent."

Why it happens

The stream id is a pure function of the run id, so a retried step reopens the same stream instead of a fresh one:

https://github.com/vercel/workflow/blob/95e292e0c1b4424998c62a134d74202c8a6539d0/packages/core/src/util.ts#L56-L66

The step-side getWritable() pipes straight into WorkflowServerWritableStream(runId, name) → world.streams:

https://github.com/vercel/workflow/blob/95e292e0c1b4424998c62a134d74202c8a6539d0/packages/core/src/step/writable-stream.ts#L107-L111

Those writes are plain side effects — nothing in the event log gates or replays them — so a re-executed step body re-emits from the top.

And the world API has nowhere to put a dedupe key:

https://github.com/vercel/workflow/blob/95e292e0c1b4424998c62a134d74202c8a6539d0/packages/world/src/interfaces.ts#L55-L60

On the wire it is PUT /v2/runs/<runId>/streams/<name> with the raw chunk as the body:

https://github.com/vercel/workflow/blob/95e292e0c1b4424998c62a134d74202c8a6539d0/packages/world-vercel/src/streamer.ts#L183-L213

Users cannot route around it, either. getWritable() inside a workflow returns only a handle — the actual IO has to happen in a step, and the step is exactly the retry unit:

https://github.com/vercel/workflow/blob/95e292e0c1b4424998c62a134d74202c8a6539d0/packages/core/src/workflow/writable-stream.ts#L4-L15

It is also reachable without a step retry

writeMulti pages at MAX_CHUNKS_PER_REQUEST and the caller re-sends the whole buffer when a later page fails. That is already acknowledged in a comment:

https://github.com/vercel/workflow/blob/95e292e0c1b4424998c62a134d74202c8a6539d0/packages/world-vercel/src/streamer.ts#L230-L237

Note: for batches spanning multiple pages, atomicity is relaxed — earlier pages may persist while a later page fails. The caller retains the full buffer on error, so chunks from successful pages will be re-sent on retry, producing duplicates. This is acceptable because the alternative (400 on all >1000 chunk flushes) is worse, and the scenario requires a network failure mid-batch.

So any flush over 1000 chunks that hits a mid-batch network failure duplicates too, without a step retry being involved.

What existing work covers, and what it does not

#2731 (with vercel/workflow-server#584) introduces framed-v2 (writerId, seq) markers and read-side dedupe. That dedupe is scoped to a single writer's lifetime — its purpose is letting a writer resend admitted-but-unconfirmed frames across a reconnect or a WebSocket → PUT fallback, per the PR: "redelivering the writer's admitted-but-unconfirmed frames first (markers dedupe any overlap)".

A retried step mints a new writer, so the overlap is not recognized and this case stays open. The markers do look like the right substrate for a fix, though: what is missing is an identity that is stable across step attempts — the way correlationId is for step events — rather than per writer instance.

Not documented

The streaming docs only state that a stream error does not trigger a producer retry:

https://github.com/vercel/workflow/blob/95e292e0c1b4424998c62a134d74202c8a6539d0/docs/content/docs/v5/foundations/streaming.mdx#L537

Nothing says that a retry for any other reason re-emits chunks. Even if the semantics stay as they are, that is worth writing down.

Origin

This came up reviewing the Python SDK's streaming port, vercel/vercel-py#256, whose description carries the caveat "Retries re-stream. A step that fails halfway already wrote what it wrote. Keep chunks idempotent." @msullivan noted it should be the framework's job and needs raising upstream:

https://github.com/vercel/vercel-py/pull/256#issuecomment-5210193290

I think that streams ought to have a way to have an idempotency key, but we'll need to raise this upstream I think. In temporal, workflows themselves can write to streams, and so they must be doing some idempotency thing for it. In workflow, the workflow needs to have a step do the [write], but the step might get retried

Filing here rather than in vercel-py because the gap is in the protocol, not in any one SDK: Python is faithfully mirroring what TypeScript does.

---

🤖 Generated with Claude Code

Start by reading packages/core/src/util.ts, packages/core/src/step/writable-stream.ts, packages/world/src/interfaces.ts, and packages/world-vercel/src/streamer.ts, then compare the framed-v2 work in #2731. Trace how retries and paged writeMulti requests resend chunks. Done should define and implement stable retry deduplication semantics, with coverage for step retries and mid-batch failures, and document the resulting behavior in streaming.mdx.