Support replay-derived step arguments without persisting full inputs
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 2.4k
- Forks
- 365
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 169
Description
Long-running workflows often reconstruct growing state from completed step results and pass that state into the next step. Even when each step returns only a small delta, Workflow persists the full input on every boundary. With a fixed amount of new state per step, the accumulated input payload grows quadratically.
let state = initialState;
while (hasMoreWork(state)) {
const delta = await advance(state);
state = applyDelta(state, delta);
}
The workflow body already recomputes arguments during replay. The missing capability is supplying selected reconstructed arguments to the executing step without persisting them again.
Proposal
Add an explicit per-step setting, following maxRetries:
async function advance(state: State, execution: ExecutionInput) {
"use step";
// Return the changes produced by this step.
}
advance.replayInputs = [0];
Indices select arguments in the original function signature. Selected arguments contain plain state data; unselected arguments, captured closures, and this retain existing persistence. The default behavior is unchanged.
Persist versioned markers and integrity fingerprints for selected inputs. Capture their values at invocation time, pass isolated materialized values to the executor in memory, and keep their contents out of durable events and queue messages. Cold execution and retries must replay to the targeted invocation, reconstruct the arguments, and verify their fingerprints before executing. Completed steps continue returning their recorded results without re-execution.
Support both Node and QuickJS. Preserve step identity, ownership, retry delays and limits, cancellation, and parallel execution semantics. Input inspection should identify values as reconstructed through replay.
Boundaries and validation
This is an input-persistence capability, not automatic state diffing or a new storage/retrieval API. Output deltas remain application-owned. Live handles and custom serialization use ordinary persisted arguments. Cold retries incur workflow replay; normal materialization still costs CPU.
Validate cold restart, interrupted execution, delayed retries, retained VMs, concurrent wakes, parallel steps, mutation after invocation, bound arguments, mixed argument modes, and mismatched fingerprints in both engines. A growing-state benchmark should show linear durable bytes when paired with fixed-size output deltas.
Related: #3026 describes accumulated-history performance costs. This proposal specifically removes repeated step-input state from durable storage and replay transfer.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the advance step entry point and trace how arguments are reconstructed during replay and passed to execution in both Node and QuickJS. Map the existing persistence, retry, queue, and event paths before designing the per-step replayInputs setting. Done means selected inputs stay out of durable events and queue messages while cold starts, retries, concurrency, mutation, and fingerprint mismatches satisfy the listed validation cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100