Feature Request: Try/Catch/Finally control blocks for workflows
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
## Feature Summary
Today a runtime failure in any operator (a Python UDF raising, a filter
referencing a missing attribute, a flaky external call) has no recovery path:
the workflow reports the error and pauses, and the only options are manual
retry or abandoning the run. There is no way to express "attempt this
subgraph; if anything in it fails, run this fallback instead", the
control-flow vocabulary has `If` for data-driven branching but nothing for
failure-driven branching.
This feature adds block-level **try/catch/finally** semantics, PL-faithful,
as two new control-block operators:
- **TryCatch**, guards a subgraph (the *try* branch). If any operator inside
it fails, the frame's input is replayed from a snapshot through the *catch*
branch. A third **Error Info** output port emits one row per caught failure
(errorType, message, operatorId, workerId) for auditing, alerting, or
routing the recovery by error type.
- **Finally**, the reconvergence point: consumes both branches' tails and
releases exactly one branch's complete output, the try side's on success,
the catch side's on failure, through the output port named for the winner
(**Try Result** / **Catch Result**), so downstream logic can also react to
the outcome itself.
Use cases: fallback models when a primary model errors, replaying input
through a cheaper/safer path, quarantining bad batches while keeping the run
alive, `catch (SpecificError)`-style routing via Error Info + If, and audit
tables of caught failures.
## Proposed Solution or Design
### Semantics
- The **try cone** (everything reachable from the Try port, up to the paired
Finally) is one *attempt*. Any failure inside it aborts the attempt: no
further user code runs on post-failure data, and the same input is replayed
through the catch cone.
- **All-or-nothing reconvergence**: Finally stages both sides and flushes
only the winner, never a mix, never duplicates. `From Catch` depends on
`From Try`, so the release decision is deterministic, never a race.
- **The outcome is part of the output**: winner rows leave through
`Try Result` or `Catch Result`; connecting both ports to one downstream
input (a union) yields "the winner, whichever it was".
- **Nesting** forms a tree: the innermost frame owns a failure; a caught
failure never leaks past its frame; a double failure (catch branch fails
too) escalates to the enclosing frame, or terminates the run with the
usual console error at top level.
- **Failures outside any frame keep today's behavior exactly** (console
error + pause, current input retriable). Frames opt their subgraphs into
drain-on-failure semantics; plans without frames are untouched.
- Infrastructure failures (worker crash, node loss, OOM-kill) are
deliberately **not** catchable, frames catch *user-code* errors.
### Mechanism (minimal engine change)
One idea carries the design: **failure becomes a dataflow event**. A failing
worker broadcasts an ordinary `State` carrying a reserved `__error__` key
(same convention as `If`'s condition State), then drains. Ports still
complete, so the stream terminates instead of hanging. Everything else is
ordinary operators plus one compiler pass, the coordinator and scheduler are
untouched:
- **TryCatch expands to two physical operators**: a *splitter* (live stream
out the Try port + an eager snapshot toward the gate) and a *catch gate*,
an `If` generalized to N conditions. The compiler pass synthesizes **signal
edges** from every try-cone tail to the gate; a dedicated
`SignalPartitioning` drops tuples at the sender, so signal edges carry only
States and end-of-stream. The gate's snapshot port *depends on* all signal
ports, which makes resolution timing structural: the snapshot is consumed
only after the try side fully resolved.
- **Per-port drain contagion**: when an `__error__` State arrives on a port,
it is delivered to the executor first (frame operators react; the default
pass-through forwards it, that is escalation), then the port is poisoned:
subsequent data is discarded without invoking the executor and its finish
hooks are suppressed, while the port still completes normally.
- **Attribution follows the failing operator**, not the data's destination:
the gate/merger configs carry their frame's cone (baked at compile time)
and act only on own-cone errors; foreign errors pass through to the
enclosing frame.
- **Wiring rules validated at compile time** with clear messages: try/catch
cones disjoint; Finally inputs must come from the paired frame's cones;
the cones may reach the post-frame region only through the Finally (no
Merger bypass); Error Info may feed the catch cone or the post-frame
region but never the frame's own try cone (structural cycle + temporal
paradox).
*(1–8 = execution order. Green = allowed external wiring; red ✕ = rejected at
compile time; dotted = synthesized signal edges; dashed = materialized
snapshot; ↔ = Error Info and catch cone may interconnect.)*
## Affected Area
- Workflow Engine (Amber)
- Workflow UI
Contributor guide
Assessment
This issue has not been assessed yet.