picatz / picatz/flowstate

Local activities: cheap effects pay a full activity round trip and three history events each — decide whether an engine-decided in-process dispatch lane earns a place, and record the answer either way

Open
#778 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design engine kind/decision
Dominant language
Go
Stars
9
Forks
0
Avg merge
3h 3m
Merged PRs (30d)
509

Description

Verified against origin/main at 41aacf2.

Problem

Every effect the durable driver runs is a full Temporal activity. All four task entry points dispatch through workflow.ExecuteActivity (pkg/flowstate/v1/engine/execute.go:845-877), ExecuteLocalActivity appears nowhere in the tree, and no doc or issue argues for or against local activities — the phrase does not occur in the repository. A regular activity costs one server round trip and three history events (Scheduled/Started/Completed) plus payload copies; a local activity runs in-process on the worker already holding the workflow task and records one marker event.

For most of what Flowstate runs that is the right trade, and the expensive half of the local-activity use case is already answered better than a local activity would: pure computation evaluates inline in workflow code (conditions, vars:, items:, most task inputs — docs/DSL.md:433-440), and the cel: task was retired precisely for refusing to pay "an activity round trip plus a history entry to compute what replay could recompute for free" (docs/DSL.md:629). What has no fast lane is the cheap effect. log (pkg/flowstate/v1/eval_task_library.go:35-41) is a sub-millisecond emit that pays the same dispatch cost as a four-minute HTTP request, and the cost lands exactly where the repo's own analysis says it hurts: history size is what forces Continue-As-New, and a 200-item loop over a one-step body already schedules 200 activities (docs/DSL.md:554-560) — ~600 history events for 200 log lines.

Two internal activities are also textbook local-activity shapes, though the saving is once per run and modest: WorkflowVars (engine/activities.go:56, one CEL evaluation carried into RunState) and CheckPlugins (engine/plugins.go:104, a comparison against a process-local catalog — which as a regular activity is answered by whichever process polls it, not necessarily the one interpreting the workflow).

Why this is a decision worth recording rather than an obvious win

A local lane pays real costs, and they are the exact machinery this engine deliberately built:

  • No heartbeat. Cancellation reaches a running activity through the heartbeat response, and WaitForCancellation: true + HeartbeatTimeout is what makes a cancelled run's wait short and honest (engine/policy.go:70-90, engine/heartbeat.go:54-72). A local activity can do neither, so only a task whose whole runtime is shorter than any cancellation anyone could care about qualifies.
  • At-least-once, batched. If a workflow task fails after a local activity completes, every local activity in that task re-executes. Only effects tolerant of duplication qualify — a repeated log line, not a repeated POST.
  • The activity-inbound guards do not run. The tenant guard is an ActivityInboundInterceptor (engine/tenant.go:135) and the task-shape policy check runs inside the activity entry points (engine/activities.go:91, checkTaskDispatchPolicy). A local entry point has to re-establish both, and the fail-closed sections of CLAUDE.md apply: a fast path that skips an admission check is a hole, not an optimization.
  • No task-queue routing. Local activities bypass queues entirely, so any task that might ever use the per-step routing docs/ARCHITECTURE.md:236 holds open can never be local. (Per-tenant routing is unaffected: the local activity runs on the worker that took the workflow task, which is already the tenant's fleet.)

Given those, "argued against, in writing" is a perfectly good outcome — the same shape as per-step queues in #133's title and the #516 record-the-answer-either-way precedent. What is not good is the current state, where the substrate's designated lane for cheap effects is simply unmentioned.

Desired outcome

A recorded decision. If adopted, the lane should be engine-decided, declared by the task, never spelled in a Flowfile: an author writing local: true on a step would be choosing durability and cancellation semantics they cannot see, the same class of knob activityOptionsFor already refuses to expose (engine/policy.go:100-110WaitForCancellation, HeartbeatTimeout, non-retryable types). No DSL change means no new abstraction, and the local driver is observably unaffected — it is already in-process — so the shared cases in pkg/flowstate/v1/tests should not move.

The declaration surface already exists. Illustrative, not the landed shape — beside NeedsPrevOutputs on TaskDef (pkg/flowstate/v1/registry.go:58,108-115):

// InProcess reports that this task is safe to dispatch as a Temporal local
// activity: sub-second, tolerant of re-execution alongside its workflow task,
// and needing no in-flight cancellation or phase reporting. Only `log`
// qualifies today; plugin tasks never do (IPC, admission, phases).
InProcess bool

and a fifth dispatch arm in executor.dispatch (engine/execute.go:859) using workflow.ExecuteLocalActivity with the step's retry policy mapped into LocalActivityOptions and checkTaskDispatchPolicy called inside the local entry point — the enumeration comment on checkTaskDispatchPolicy (engine/activities.go:72-90) already prescribes exactly this step for any new entry point.

Acceptance criteria

  • The decision is written down either way — here, and as a row or a refusal note in docs/ARCHITECTURE.md's "Leaning into Temporal" table.
  • If built: log dispatches locally on the durable driver; the driver-agreement suite in pkg/flowstate/v1/tests is unchanged and green on both drivers; a test asserts the task-shape policy check still denies on the local path (the negative direction, per CLAUDE.md's isolation-test rule); history for an N-step log workflow shrinks from ~3N activity events to N markers.
  • Nothing that can heartbeat, route, or be non-idempotent is ever eligible; plugin tasks remain regular activities.

Generated by Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the dispatch paths in pkg/flowstate/v1/engine/execute.go, the TaskDef declaration in pkg/flowstate/v1/registry.go, and the entry-point policy checks in engine/activities.go. Review the cancellation, heartbeat, routing, and admission constraints, then inspect docs/ARCHITECTURE.md and the shared driver tests. Done means the local-activity decision is recorded with its rationale, and, if adopted, the stated acceptance tests and safety constraints are met.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.