picatz / picatz/flowstate

decision: the language cannot name a reusable computation — `vars:` are values, `cel.bind` is one expression, `call:` is a whole run; a `functions:` block compiled by inlining keeps the IR plain `2026.1` CEL and both drivers untouched, and plugin- or operator-defined CEL functions stay refused

Open
#1,871 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design dsl
Dominant language
Go
Stars
9
Forks
0
Avg merge
3h 3m
Merged PRs (30d)
509

Description

Written against 7530b29. A language decision that the profile, the catalog and the optimizer work all wait on.

Problem

Flowstate positions the Flowfile as a language that "compiles to a typed Protobuf specification" and CEL as its one expression dialect. The dialect has no way to name a reusable computation:

  • vars: are values, not functions: a workflow-level var "may reference nothing at all", a var "may not read its siblings" (docs/DSL.md:1166-1200), and a step var is scoped to one step.
  • cel.bind names a value inside one expression (docs/DSL.md:390), and a two-variable comprehension is a body, not a definition.
  • call: reuses a whole workflow, with a durable child run, a spec, and an input/output contract — the right tool for a process, and a heavy one for ${normalizeEmail(x)}.
  • First-party macros (sum, reduce in celfold.go:131; flowstate.@orderedMap in celmap.go) show the mechanism the tree already uses for house vocabulary: a parse-time expansion that keeps the stored IR plain CEL.

The consequence is visible in real files: the same ${x.trim().lowerAscii().replace(" ", "-")} chain appears in step after step, a policy predicate is copied between a switch: and an if:, and an organization that wants isBusinessDay(ts) has no spelling for it short of a callee workflow. Every other language Flowstate is compared with in docs/COMPARISON.md has a function definition.

The extensibility half of the same question: plugins evaluate no CEL (#1489, checked and not filed) and cannot register functions, and #1861 is deciding how first-party names are spelled. Whether a user may add a name — in a file, in an operator's profile, or never — is not written down anywhere.

Options

  1. A functions: block in the Flowfile — named CEL lambdas with declared parameter types, functions: { slug: { params: {s: string}, returns: string, body: ${s.trim().lowerAscii().replace(" ", "-")} } }, used as ${slug(inputs.title)}. Compiled by inlining (cel-go's NewInliningOptimizer, cel/inlining.go:70, which wraps a multiply-referenced argument in cel.bind) so the stored IR contains no new call and both drivers execute unchanged CEL. Recursion is refused at compile (the inliner cannot express it, which is the bound). Type-checked at the definition against the profile, so an error names the definition once rather than every use.
  2. File-scoped macros — the same block registered as cel.Macros at parse time, the way sum/reduce are. Same IR outcome, worse diagnostics (a macro expansion error points inside the expansion), and fmt cannot write the expansion back (marshal.go:900-909).
  3. Refuse, and say so — reuse is call:; the DSL records the decision, and hover/flow lint suggest call: when the same expression appears N times.
  4. Operator-defined functions (a profile extension loaded by the worker) — refused in any option: a function the spec does not carry would make the same spec mean different things on different workers, which invariant 3 forbids and Worker Versioning cannot see.

Recommendation: option 1, because it is the only shape where the definition is in the spec (proto-first, replayable, attested), the runtime is untouched (inlining happens in the compiler), and every tooling surface already has a place for it (hover from the definition's own prose, completion from the file's function list, flow tasks-style listing, the LSP's rename from #1435).

Desired outcome

A decision recorded in docs/DSL.md with the invariant it implies. If option 1: the grammar, the proto (Workflow.functions as a message, so buf breaking guards it), the inlining pass in the compiler, the type rule (a parameter is typed; dyn allowed and discouraged), the recursion refusal, and the tooling rows above. Plugins stay CEL-free in every option; #1861 gains the sentence "user-defined names are lowerCamel and may not shadow a profile name".

Acceptance criteria

  • A Flowfile defining slug and using it in three steps compiles to a spec whose expressions contain no slug call and evaluate identically on both drivers (conformance case).
  • A definition that calls itself, or two that call each other, is refused at validate with a position on the definition.
  • A definition whose body does not type-check is reported once, at the definition.
  • fmt writes the file back with the definitions intact and the uses unexpanded (the source form is the file; the spec is the expansion).
  • Hover on slug( shows the definition's declared parameters and its description:; flow tasks-class listing shows the file's functions.

Constraints

  • No new evaluator, dispatcher or runtime function: the runtime never sees a user-defined name (invariant 2 and 4). This is what makes option 1 safe under Worker Versioning.
  • Bounded: a cap on definitions per file, body size, and inlined expansion size, so a function used inside a loop body cannot multiply the spec past the size bounds size.go already states.
  • The stored IR stays 2026.1 CEL; a spec compiled with functions is readable by a worker that predates the feature, which is the reason to inline rather than to extend the runtime.

Open questions

  1. May a definition reference inputs.* or vars.* directly (a closure over the workflow's roots), or only its parameters? Parameters only is smaller and composes; closures are what vars: already is.
  2. Do definitions cross a call: boundary (a library workflow exporting functions)? Not in the first cut: #106's module design owns exports.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/DSL.md and the existing macro implementations in celfold.go:131 and celmap.go, then read cel/inlining.go:70 and marshal.go:900-909. Trace how definitions would enter the profile, compiler, proto, formatter, and tooling surfaces without changing either driver. Done means the decision and invariant are documented, with the listed validation, conformance, formatting, hover, and listing cases covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
compilers, developer-experience, documentation, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.