Infrequent polling has no spelling: an unmet `expect:` is permanently non-retryable, so every poll pays history per probe through `loop:`

Verified against origin/main at 41aacf2.

Problem

Temporal documents two shapes for polling an external system until it reports done (design-patterns/polling). The frequent shape (a heartbeating loop inside one activity) and the periodic-sequence shape (a workflow loop bounded by Continue-As-New) are both reachable from a Flowfile today. The infrequent shape — a single-probe activity that fails while the condition is unmet and lets the server's RetryPolicy re-run it on a fixed interval, so that probes add zero events to workflow history — is not reachable, and the reason is a deliberate rule made for a different case.

The chain, each link verified:

• The http task's expect: failing returns ErrorKindInvalidInput (pkg/flowstate/v1/eval_task_http.go:673-675).
• ErrorKindInvalidInput is in PermanentErrorKinds() (pkg/flowstate/v1/errors.go:94-103), which becomes every activity's NonRetryableErrorTypes (pkg/flowstate/v1/engine/policy.go:87,168-175).
• That list is never overridable by a step's own policy, on purpose (pkg/flowstate/v1/engine/policy.go:95-99): "whether a failure can succeed on another attempt is a property of the failure, not a preference."
• The permanence of an unmet expectation is itself documented as deliberate (pkg/flowstate/v1/eval_task_http.go:614-618): "the author has described what success looks like, and a response that does not match it is this endpoint answering this request in a way they said is wrong — repeating the request will not change their mind. An expression that needs a retry says so by accepting the retryable status instead."

That rationale is right for a wrong answer and wrong for a pending one. A 200 whose body says {"status": "running"} is exactly a response that will change its mind — that is the definition of a poll — and "accept the retryable status instead" cannot express it, because the response is not a retryable status: it is a success whose body is not yet the one wanted. So the only spelling for "ask until it says done" is loop: + sleep: (the documented one — examples/loop-poll-until/workflow.yaml), and every probe in that spelling costs an activity round trip plus a timer in history, bounded only by the engine's automatic Continue-As-New (pkg/flowstate/v1/engine/execute.go:1567-1575).

Concretely: polling a job every 60s for a day is 1,440 probes. As a loop: that is ~1,440 activity round trips and timers written to history, across several Continue-As-News, each re-carrying the run's whole state. As activity retries it is one pending activity with an attempt counter, no history growth at all, and the interval survives worker restarts because the server owns it. This is the pattern Temporal names as the recommended one for intervals of a minute or more, and it is the one the DSL cannot say.

What already exists (the spellings this would compose with, not replace)

• RetryPolicy already expresses exactly the needed schedule: backoff_coefficient is validated gte: 1.0 and 1.0 is fixed-interval (proto/flowstate/v1/workflow.proto:1616-1633); activityOptionsFor maps it onto Temporal's policy per step (pkg/flowstate/v1/engine/policy.go:111).
• Every task activity already heartbeats with a phase (pkg/flowstate/v1/engine/heartbeat.go, policy.go:81-84), and the server already reads heartbeat details into the pending view (pkg/flowstate/v1/server/server.go:1801) — so a retry-driven poll stays observable (pending activity, attempt count, phase) rather than going dark between probes.
• The http task already has the machinery for a deferred response-scope expression: expect is a DeferredInputs/ExpressionInputs member evaluated against response (pkg/flowstate/v1/eval_task_http_def.go:72,85); a sibling input reuses all of it.
• loop: stays the right spelling for pagination, for polls that accumulate state, and for polls whose probe is itself multi-step — nothing here touches it. examples/loop-poll-until would gain one paragraph pointing at the split.

Nothing in the tree spells "this response is not yet the one wanted, ask again later" today — I checked expect, tolerate/continue_on_error, RetryPolicy, and the loop docs; that absence is the finding.

Sketch

Illustrative, not the landed shape — the crisp split being proposed is: expect: says this response is wrong (permanent, exactly as documented today), and a new input says this response is not yet (retryable, driven by the step's ordinary retry:):

- id: probe
  http:
    method: GET
    url: https://api.example.com/jobs/${inputs.job}
    until: ${response.json.status == "done"}
  retry:
    max_attempts: 120          # give-up bound, the role max_iterations: plays in a loop
    initial_interval: 60s
    backoff_coefficient: 1     # fixed interval: Temporal's infrequent-polling shape

An unmet until: fails the attempt with a retryable kind (a new ErrorKind, not ErrorKindInvalidInput, so the never-overridable non-retryable list stays exactly as it is); exhausting retry: fails the step with a message naming the condition that never held, the same distinct-failure stance LoopIterationLimitError takes. expect: and until: on one step compose: expect: judges every response first (a 500 or a wrong shape is still wrong), until: then decides pending-versus-done.

Constraints

• Both drivers must agree: the local driver already honors RetryPolicy including intervals, so the shared cases in pkg/flowstate/v1/tests need the poll shape added and called from both (CLAUDE.md, "Both execution drivers must agree").
• The non-retryable list stays non-overridable; this adds a kind that was never in it, it does not open the list.
• flow validate should refuse until: with max_attempts: 1 (a poll that cannot poll), the same class of diagnostic as a literal expect:.
• Naming: until: collides conceptually with the loop's until: — deliberately, it means the same thing ("stop condition over what the body/response reports"), but if that reads as confusing, retry_until: is the fallback spelling; the issue is the capability, not the keyword.

Cost

A second expectation-shaped input on the http task is a real cost: two adjacent inputs whose difference (wrong vs. not-yet) has to be taught. The alternative costs more: an opt-in bool beside expect: couples one expression to two meanings, and doing nothing leaves the DSL paying O(probes) history for a pattern the substrate offers at O(1).

Acceptance criteria

• A Flowfile can express "probe this endpoint until its body says done, every N seconds, giving up after M attempts" such that an unmet probe adds no workflow history events (one pending activity, server-side retry).
• Local and durable drivers report the same outcome for met, eventually-met, and never-met polls (shared cases in pkg/flowstate/v1/tests, called by both drivers).
• flow validate accepts it; an example under examples/ exercises it; examples/loop-poll-until/README.md says when to use which shape.
• The eval_task_http.go:614 comment is updated to name the new spelling as where "needs a retry" now points for body conditions.

Generated by Claude Code

Start with pkg/flowstate/v1/eval_task_http.go, engine/policy.go, and the shared cases in pkg/flowstate/v1/tests; compare both execution drivers and the existing loop-poll-until example. Trace how RetryPolicy and non-retryable errors are validated and reported. Done means an example and validation support the new polling shape, both drivers agree on met, eventual, and exhausted polls, and the loop documentation explains when to use each form.