picatz / picatz/flowstate

Infrastructure topology: state, sharding, placement, and federation — which layer answers what

Open
#185 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

deployment design kind/design-record
Dominant language
Go
Stars
9
Forks
0
Avg merge
3h 3m
Merged PRs (30d)
509

Description

The rule this issue exists to write down: flowstate is deliberately stateless above Temporal, and every persistence/scale question must prove it cannot ride Temporal's machinery before it earns new machinery. Make Temporal carry it first.

State: where it lives today, and what earns a store later

Today everything durable is history, memos, and search attributes; the server holds nothing. Our scaling story IS Temporal's proven one (sharded history, visibility stores), and every piece of state kept out of our own database is state we never shard, back up, or fail over. The three needs that will eventually earn their own store — the content-addressed value store (#149 scale addendum), the replay corpus at scale (#182), a module/type registry (#177 team sharing) — are all content-addressed and immutable: the EASY kind of database (digest-keyed, no coordination, cache-friendly), pluggable over S3/GCS from day one rather than a bespoke engine. Anything mutable keeps riding Temporal until proven impossible.

Sharding and placement of WORK

The primitive already exists: Temporal task queues. runs_on: pools (#156) map onto queues; worker capabilities decide what runs where; per-tenant namespaces are the isolation seam. The isolation ladder (subprocess → runc → gVisor → microVM) slots UNDER pools as capability, not beside them. Nothing new to invent — the design work is the pool/capability vocabulary in the schema and honest docs about what placement can and cannot promise per deployment.

Deployment shapes

k8s / EC2 / Cloud Run are shapes of the same three binaries (server, worker, Temporal). The k8s plugin tracks both directions: deploy onto, control from outside (#180 stage 2). The observability lab is the local shape. Each shape documented as a recipe, none privileged in code.

Federation

The workload-identity story extended (#115 JIT federation, #173 ladder): clusters trust each other's short-lived identity attestations rather than sharing credentials — the WIF shape, cluster-to-cluster. Cross-cluster workload placement composes from: digest-pinned specs (they travel whole), identity attestation, and per-cluster policy that stays sovereign — a receiving cluster's egress/secret/authz rules apply to arriving work exactly as to local work, no imported authority.

Harmony claim, kept honest

Temporal owns durable state. Flowstate owns meaning and policy. The platform owns placement. Every addition gets tested against "which layer already solves this" — and "flowstate needs its own database" requires naming the property (content-addressing, cross-run sharing) that history/memo/visibility structurally cannot provide.

Related: #149, #156, #173, #177, #180, #182.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked issues #149, #156, #173, #177, #180, and #182 to understand the existing decisions around state, placement, deployment, and federation. The documentation is done when it clearly assigns responsibilities to Temporal, flowstate, and the platform without introducing an unscoped implementation task.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems, documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.