picatz / picatz/flowstate

decision: a task is the unit of recovery, and nothing tells an author which operations are safe as one task, which need a resumable protocol, and what to write after `UpstreamUnknown` — the boundary contract before the "durable agents" story widens

Open
#1,879 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design dsl engine kind/decision
Dominant language
Go
Stars
9
Forks
0
Avg merge
3h 3m
Merged PRs (30d)
509

Description

Observed behavior

At 7530b29, the engine's recovery unit is the activity. A heartbeat carries the closed Phase vocabulary and nothing else (pkg/flowstate/v1/engine/heartbeat.go:133); a retried attempt starts from zero, which #1516 records and proposes an opt-in checkpoint for. An operation whose response is lost is classified UpstreamUnknown and is permanent by design (pkg/flowstate/v1/errors.go:48-54: "The step fails once and the author decides, which they can do and the engine cannot"). The http task exposes retry_on_unknown_outcome for the author who knows the operation is safe to repeat (docs/reference/tasks.md:43), and a plugin can answer unknown_outcome (plugin.proto, ExecuteResponse).

What the author is supposed to decide is written nowhere they read. grep -rn -i "unknown_outcome\|UpstreamUnknown" docs/*.md finds the one generated table row. docs/DSL.md and docs/PLUGINS.md do not have a section on it. Three examples show the pattern by hand (examples/deployment-reconciler sets the flag because "setting an exact count is safe to repeat"; examples/agentic-loop leaves it off because "the gateway may well have run the turn and charged for it"; examples/plugins/github/issue-comment.yaml), and each reasons from scratch in comments.

Meanwhile the surface the product is being pointed at is long operations: docs/VISION.md names "agentic investigations", "a coding agent (Claude Code, Codex) as a step a durable workflow drives, retries, and gates"; codex.exec is a task; git.log and github.*_list paginate inside one task. Durable orchestration around such a task does not make the task's internal progress recoverable, and a checkpoint (#1516) cannot make an external effect and its progress record atomic: the effect can land and the checkpoint be lost, or the reverse, on every crash. The honest contract therefore has three classes, and today the author is left to discover which one they are in:

  1. Small idempotent tasks. The effect is safe to repeat (an exact-count set, a PUT, a request with an idempotency key the provider honours). Retry is correct, including after an unknown outcome. #1677's call identity is what makes the key available.
  2. Resumable protocols. The operation is long and has natural progress (pages, events, recipients). A checkpoint lets the next attempt resume, and the author accepts that the last unit before a crash may repeat. #1516.
  3. Reconciliation after an unknown outcome. The effect is not safe to repeat and has no resumable structure (a charge, a deploy, a message). The correct next step is a read: ask the provider what happened, then branch. Today that is a continue_on_error step followed by a read step guarded on ${steps.x.error.kind == "UpstreamUnknown"}, which the examples reinvent and no document names.

Desired outcome

A recorded decision, then the smallest surface it implies:

  • The contract, in docs/DSL.md and docs/PLUGINS.md. One section naming the three classes, how an author recognizes which one a task is in, what to write for each, and what the engine guarantees for each: at-least-once for class 1, resume-from-checkpoint with a repeated tail for class 2, exactly-once knowledge (not exactly-once effect) for class 3 via a read. Cite the examples as the worked cases and remove the per-example re-derivation.
  • A task's class is a declared fact, not a guess. A plugin descriptor states whether an effect is idempotent, resumable, or neither (the effect tier #239 and the claims digest #712 already carry "writes"; this is the retry-safety axis of the same fact), so flow validate can warn when retry_on_unknown_outcome: true is set on a task that declared its effect unsafe, and flow lint can suggest the reconcile pattern when a class-3 task has no read after it.
  • The reconcile pattern gets a spelling, if the record decides it earns one. Candidates: a documented idiom over existing keys (no language change), or an on_unknown: handler beside retry: that names the read step. The record should prefer the idiom unless the lint above cannot express it.
  • Scope the agent story to the contract. docs/VISION.md's agentic paragraph says which class codex.exec and an LLM turn are in today (class 3 with a checkpoint pending) and what an author gets, so "durable agents" means what the engine delivers.

Acceptance criteria

  • The DSL section exists, and each of the three examples links it instead of re-arguing it in comments.
  • A conformance case per class on both drivers: an idempotent task retried after a lost response performs its effect twice with identical output; a checkpointed task resumes from its page; a class-3 task fails once with UpstreamUnknown, the read step observes the effect, and the run branches on it. The effect counts are asserted, which needs the ledger #1864 proposes or a per-test stand-in.
  • flow validate warns on retry_on_unknown_outcome: true against a task whose descriptor declares a non-idempotent effect, with the descriptor field named.

Constraints and dependencies

  • Nothing here promises exactly-once effects; #713's non-goals rule that out, and the contract's value is in saying so per class.
  • A declared retry-safety claim informs; operator policy enforces (#239's rule that a plugin cannot vote itself a tier).
  • Adjacent: #1516 (checkpoints, class 2), #1677 (call identity and idempotency keys, class 1), #1864 (effect ledger), #1490 (a panic mid-effect), #239 and #712 (effect tiers and claims), #1133 (resumable codex threads), docs/VISION.md.

Open questions

  • Whether a class-3 read step should be able to adopt the effect's result as the failed step's outputs (so downstream steps see one value regardless of which attempt produced it), or whether the read is always its own step. The narrower answer is the safer first slice.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/DSL.md, docs/PLUGINS.md, docs/VISION.md, and the three referenced examples to compare the existing explanations. Then inspect pkg/flowstate/v1/errors.go, heartbeat.go, the HTTP task, plugin.proto, and current validation and driver tests. Done means the contract and examples are updated, the three conformance cases are covered, and validation warns for unsafe retry_on_unknown_outcome usage.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, documentation, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.