KeeperHub / KeeperHub/keeperhub
Nothing a workflow computes survives the run: no workflow-scoped key-value state
- Dominant language
- TypeScript
- Stars
- 24
- Forks
- 93
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 266
Description
Part of #2293. Read the design questions there before starting.
## Reason
Nothing a workflow computes survives the run. There is no workflow-scoped key-value store anywhere in the schema, so a monitoring workflow that needs a cursor - "the last block I scanned", "the transactions I have already alerted on" - has two options:
1. Re-scan a rolling time window on every run and re-derive what is new. This has a visible failure mode: a resolved alert can reopen when the same underlying transaction is still inside the window on the next pass.
2. Provision an external database and reach it through the Database Query node. That works, and it is disproportionate for one key-value pair.
Most workflow platforms ship a small built-in store for exactly this reason. KeeperHub does not have one.
## Scope
Two actions, workflow-scoped:
- `state/get` - `{ key }` returns `{ value, exists }`
- `state/set` - `{ key, value, ttl? }` returns `{ success }`
Backed by storage KeeperHub already operates, so the user provisions nothing.
## What a solution needs to cover
- **Scope and isolation.** Keyed by workflow, isolated per organization. A workflow must not be able to read another workflow's state, and the key namespace must not be shared across organizations.
- **Concurrency.** Two executions of the same workflow can overlap. Read-modify-write on the same key will race. Either the operations are atomic, or the race is documented and the API offers something that is (compare-and-set, or an atomic increment).
- **Size and count limits, enforced.** This is a per-workflow store, not a database. An unbounded store attached to every workflow is a storage-growth problem with no natural ceiling.
- **TTL and eviction**, so abandoned keys do not accumulate forever.
- **Deletion semantics.** What happens to a workflow's state when the workflow is deleted, duplicated, or exported and imported elsewhere.
- **Visibility.** Whether the current state is inspectable from the UI. Debugging a monitor with an opaque cursor is unpleasant; this does not have to ship in the first version, but the storage choice should not rule it out.
## Note on the storage choice
The title of the internal ticket behind this says "serverless-first, DB as fallback", and that framing is worth keeping: the point is that the user provisions nothing. Whether the backing store is Postgres or Redis is an implementation decision, and it should be argued from the concurrency and eviction requirements above rather than assumed.
## Out of scope
Cross-workflow shared state, and state readable from outside an execution through the API. Both are reasonable later; neither should be designed into a first version.
---
Tracking: [KEEP-1036](https://linear.app/keeperhubapp/issue/KEEP-1036) (internal tracker, not publicly accessible)
Contributor guide
Research direction
Start by reading the design questions in #2293, which this issue identifies as prerequisite context. No source files, tests, or implementation entry points are named. Done means defining and implementing workflow-scoped get/set actions with isolation, concurrency behavior, limits, TTL and lifecycle semantics, while keeping cross-workflow and external API access out of scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, backend-api-design, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100