Execution surfaces record: artifacts, agent steps, isolation, plans, console
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 9
- Forks
- 0
- Avg merge
- 3h 3m
- Merged PRs (30d)
- 509
Description
Companion to the identity record (#337) and the pillar map (#336). Like #337, this is a
tracking record: it names the workstreams, grounds each against what the tree already
has, and states the invariants they must hold to. Slices get their own issues as they
start; nothing here is committed to until a slice lands under the house gate (a Flowfile
expresses it, flow validate accepts it, an example in examples/ exercises it in CI).
The organizing idea
Everything below is one sentence applied to new ground: the file says what should flow,
and the deployment decides as whom, with what, within which bounds, on which compute.
Durability, identity, policy, isolation, and preview are properties of the deployment,
not problems of the author. That is already how secrets, egress, dispatch policy, and
identity work here; these workstreams extend the same discipline to data, long-running
loops, sandboxed tasks, side-effect preview, and the screens an operator approves from.
Invariants these workstreams add
- Enforcement lives outside the bounded thing. The plugin doctrine (a plugin never
participates in its own policing) generalizes: an isolation tier's controls, meaning
egress, filesystem, resources, and secret reach, are enforced at or outside the tier
boundary, never inside it. Today's egress policy is enforced inside the Gohttp
task, which is correct while every task is our code. The moment a task is a container
running arbitrary code, in-process enforcement is theater: the code opens its own
sockets. For contained tiers, the same policy file compiles to boundary enforcement
(network namespace plus enforcing proxy, or VM network configuration), and a tier
that cannot enforce the declared policy refuses to dispatch rather than degrade. - Two agent postures, named. Transparent loop: the Flowfile owns the tool-calling
loop, every tool call is a step dispatch, the full policy spine applies per call, and
the trajectory is the workflow history. Opaque delegate: a whole agent runs as one
sandboxed task (the codex plugin's shape), bounded by tier, egress, and operator base
config, with its trajectory internal. Both are legitimate and they are different
trust postures; a deployment may allow one and not the other by task-shape policy
(#187). Choosing between them is a security decision, so the docs must name them. - Static graph, dynamic trajectory. The compiled spec stays frozen. Loops,
branches, and policy-gated computedcall:targets (#174) make the trajectory
dynamic; runtime graph mutation is off the table, because the frozen spec is the
audit story. - Budgets are deny-by-default resources.
loop:already refuses to compile without
max_iterations:and fails distinctly on exhaustion (LoopIterationLimitError). New
budget dimensions (wall time, tokens, cost) follow that shape: bounded or it does not
compile, exhaustion a first-class outcome a file can handle, consumption observable. - References, not material, third application. Secrets ship this way, credentials
are specified this way (#337), and artifacts follow: run state carries
content-addressed references, material moves inside activities, retention is
deployment config.
Workstreams
A. Artifacts and cache. A content-addressed artifact store behind a pluggable
backend (local directory first; object-store and registry backends later, aligned with
G), ${artifact(...)} references in state, producing steps declaring outputs,
consuming steps resolving in-activity. Sizes and digests are observable to policy;
contents never enter history. The substrate's per-event and per-payload limits make
this a correctness requirement rather than a preference, the same reason loop: owns
Continue-As-New. cache: is the sibling: keyed restore/save with CEL-derived keys.
Open questions: manifest schema (proto-first), garbage collection ownership, tenant
scoping (default yes), flow test stubbing. This is the prerequisite for build shapes,
data pipelines, agent transcripts, and plan documents (D).
B. Budget dimensions beyond iterations. loop: has the iteration ceiling; add wall
time and, once an llm task exists, token and cost dimensions with the same two
properties the iteration bound has: compile-refused when absent where required, and
exhaustion a distinct, handleable outcome. Consumption events feed the same
observability the run already emits, so deployment rules can bound cumulative spend.
C. The llm task family and MCP tool sources. The transparent-loop enablement: an
llm task (completion in; message and tool-call requests out) with providers as
ordinary task plugins over the existing Connect plugin protocol, no new plugin kind.
An mcp: tool-source configuration so a declared MCP server's tools become
dispatchable steps, every invocation crossing task dispatch and therefore task-shape
policy, egress, and brokered credentials (#337 B). The loop itself is loop: + if:
in the file; no agent: macro until worked examples prove the boilerplate is real, and
then only as sugar compiling to the same spec. v1 is final-only responses (durably
honest), tool schemas derived from task and plugin manifests with typed inputs (#177)
making them precise, transcripts stored as artifacts (A) with digests in state. The
differentiated half is testing: flow test with scripted LLM responses, stubbed tools,
and the virtual clock gives deterministic agent-trajectory tests.
D. Plan capability and side-effect governance. Write-capable tasks may declare a
plan capability in the plugin protocol, the structured diff of what the task would
do, and flow run --plan composes declared plans into a plan document (an artifact).
An approval gate binds to the plan digest; apply refuses on divergence between the
approved plan and dispatch-time reality. One mechanism reviews a deploy, a data
migration, a reconciliation, or an agent's proposed batch of calls. Batch approval is
what saves a human from per-call fatigue in a transparent loop. Plan fidelity is a
provider claim, not an engine guarantee, and the docs say so.
E. Isolation tiers. Tier vocabulary as deployment config: in-process, subprocess,
container, and stronger sandboxes, with per-task minimums set by deployment policy
(#187's shape) and plugins declaring the tier they require. Invariant 1 above is the
heart, and the honest v1 is subprocess + container with a boundary egress proxy
evaluating the same CEL policy file (deny-wins, redirect re-checking, TLS floor
preserved). Container images are pinned and verified under the same distribution
doctrine as plugins (#146). Opaque delegates (#162's generalization) run in these
tiers with broker-minted per-invocation credentials the agent never holds.
F. Worker pools and federated compute. Routing steps to worker pools with declared
capabilities (GPU, OS, in-VPC, tenant-dedicated): surfacing task queues with Flowstate
vocabulary, deployment-versioning discipline already in place. A worker is a workload
with an identity (#337); pool admission is CEL over its claims; pull-only. Capability
taxonomy is data (labels and resources matched by CEL), not logic. flow run local
ignores routing with a stated notice, per the rehearsal-honesty rule.
G. Distribution as registry artifacts. #146 named discovery-by-path as the mutable
reference problem. The answer: plugins, compiled specs, and worker images as signed,
digest-pinned registry artifacts, with call: able to reference a workflow by digest,
extending the author pin from #338 from files to registries. Provenance attestation is
in scope; the long game is runs producing attested artifacts about themselves through
the run-identity issuer (#337).
H. Console and approval inbox. One binary; flow server serves it. Run explorer
over the existing list/get/watch RPCs and CEL filters, live progress, policy-denial
browser over structured denial events, budget views (B), and above all the approval
inbox: OIDC login, pending gates scoped to what the viewer's identity may satisfy, an
approve click that is the attested signal satisfying signals: and its
distinct-from-starter rules. Authorization for every view is the same RPC authz
surface; there is no console-only permission model, ever.
I. Role bundles and flow policy explain. The model exists (identity plus CEL
rules everywhere); what teams need is legibility. Named roles as curated, reviewed rule
bundles mapped from IdP groups: sugar compiling to the same rules, never a second
model. And a flow policy explain --as <identity> that answers "what may this
identity do here", which the closed task registry and typed rules make computable.
Sequencing
A (artifacts/cache) first, as the prerequisite for C's transcripts, D's plan documents,
and G's registry story. B and C together as the agent-loop slice, shipped with the
deterministic flow test story front and center. E's subprocess/container tier with
boundary egress before any opaque-delegate claims widen. D, F, G, H, I as they earn
slices. Each lands under the house gate or not at all.
Anti-goals
Not a serving substrate: work here finishes or reconciles, and scheduling onto a
cluster is the cluster's job. Not infrastructure state ownership: plan/apply
governance yes; resource graphs and state files belong to providers, as artifacts or
in git. No runtime graph mutation. Not a second agent framework: the llm task,
loop:, and the policy spine are the framework; no bundled prompt library, no
opinionated planner. Opaque delegates exist precisely so people bring the agent they
already use. Nothing unmetered: no unbounded loops, no unbounded history, no invisible
cost. The constraint is the product.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the sequencing section, then inspect the existing loop:, flow validate, Flowfile, and examples/ paths before choosing a separately scoped slice. The issue names broad workstreams rather than a file or test, so confirm the slice's acceptance criteria with the maintainers. Done means a bounded implementation lands under the house gate: a Flowfile expresses it, flow validate accepts it, and an example exercises it in CI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100