Design: a workspace substrate — the thing steps need to hand each other more than a value

Filed out of #149, where it surfaced as the reason commit_push cannot decompose into stage: / commit: / push:. It is a much larger gap than that one plugin, and it is worth naming on its own rather than letting each plugin invent a private answer.

The gap

Steps hand each other values. steps.<id>.<output> is a protobuf Value, it lives in RunState, and it is size-bounded on purpose (CheckRunStateSize) because everything in it is durable history. That is exactly right for a URL, a status, a decision, a list of ids.

It is wrong for a tree. Today a workflow cannot express:

• clone a repository once, then run three analyses over the same checkout
• build a binary in one step and test it in the next
• download a dataset, then fan out map work across its partitions, then reduce
• stage files, commit them, push them — the #149 case

Each of those wants one step to produce bytes at scale and a later step to consume them, and the only channel between steps is a durable value that must stay small. So every plugin that needs a tree does the whole job inside a single activity, and the composition the DSL otherwise offers stops at the plugin boundary. commit_push is the honest version of that compromise; a less careful plugin would have shipped the broken split.

This also blocks the map-reduce and fan-out shapes recorded in #166 in their realistic form. Fanning out over a list of ids works today. Fanning out over partitions of a large intermediate result does not, because the intermediate cannot exist.

What makes this hard, and why it is a platform primitive

A workspace is not a temp directory. To be usable by a durable engine it has to answer, at minimum:

• Replay. A run that resumes on another worker must find the same content, or the driver-agreement invariant is dead. So the reference in history is content-addressed and immutable; the bytes live outside history.
• Both drivers. flow run local and the Temporal driver must agree on what a workspace is, or local runs stop predicting production — which is the entire reason local execution exists.
• Bounds. Total size, per-entry size, entry count, and lifetime, each bounded independently, because they are different resources and an attacker picks which one to attack. The alias-expansion lesson applies directly: a bound on one does not bound another.
• Lifetime and GC. A run that fails, is cancelled, or is terminated must not leak its workspace. That is a reclamation problem with a correctness requirement, not a cleanup nicety.
• Isolation. Tenant boundaries, and the negative direction tested (A cannot reach B, not A reaches A) — a workspace handle from one run must be unusable from another, and the encoding must make forgery impossible rather than merely unlikely, per the namespacing lesson.
• Secrets. A workspace is a place bytes accumulate, which makes it a place secrets accumulate. Whatever the containment story is, it needs the same shape-tests the rest of the secret work has.

Shape sketches, none chosen

1. Content-addressed blob store with handles in RunState. A step returns a digest; a later step declares it needs it and the runtime materializes it. Replay-safe by construction. Needs a store (local FS for the local driver, object storage for durable) and a GC.
2. Explicit workspace: on a group of steps. Scoped like a loop body: steps inside share a tree, the tree is named in the DSL, and it does not escape. Ergonomically closest to what people ask for; hardest to make replay-safe, because it implies mutation.
3. Nothing — keep collapsing into one activity. The status quo. Worth stating as a real option: it is what every plugin does today, it is correct, and its cost is only that composition stops at the plugin boundary. If the answer to this issue is "1 and 2 are not worth their weight," that is a legitimate outcome and this issue should be closed saying so.

Not this yet

No implementation. This is a design issue, and the deliberate question in front of it is whether option 3 is actually fine for another few months. The trigger to decide is the first use case where collapsing into one activity is not honest — a build/test split, or a fan-out over a large intermediate. Neither exists in examples/ today.

Related: #149 (where this surfaced), #166 (fan-out/fan-in shapes that want it), #95 (the plugins most likely to need it).

Start with the workspace gap described around RunState, CheckRunStateSize, and the flow run local entry point, then compare the local and Temporal driver requirements. Review examples/ and related issues #149 and #166 to identify a concrete triggering use case. Done means choosing or rejecting a workspace model and documenting its replay, bounds, lifetime, isolation, and secret-handling guarantees.