KeeperHub / KeeperHub/keeperhub
feat: add a Wait/Delay node backed by the Workflow DevKit's durable sleep primitive
- Dominant language
- TypeScript
- Stars
- 24
- Forks
- 93
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 266
Description
### Before filing
- [x] I searched open and closed issues for this proposal.
- [x] I checked the docs and the current behaviour on `staging`.
- [x] This is one change, not several.
### Reason: what you cannot do today
No wait/delay/pause node exists anywhere in the workflow builder. Grepped `plugins/`, `lib/steps/`, and `SYSTEM_ACTIONS` in `lib/mcp/workflow-schema-constants.ts` for `wait`/`delay`/`sleep`/`pause` — zero hits. Every node in a workflow runs to completion in one pass; there is no way to say "pause here for N hours" or "wait until ``" inside a workflow.
The dependency that would back this already exists: `workflow`/`@workflow/core` (Vercel's Workflow DevKit, already a `package.json` dependency) ships a durable `sleep(duration | Date | ms)` primitive — `@workflow/core`'s `dist/sleep.d.ts:12-32`, documented as "a built-in runtime function that uses timer events in the event log," i.e. it checkpoints and resumes without holding a running process. It is imported into generated export code (`lib/workflow/codegen/sdk.ts:252` unconditionally imports `sleep` from `workflow`) but is never actually invoked anywhere in the codebase.
### Reason: what the workaround costs
A monitor that needs a delay — "wait 24h after an alert before re-checking," "wait out a cooldown period between two on-chain actions" — has to either re-trigger externally via a separate cron/schedule trigger and re-derive state on every run, or manually chain multiple workflows together via webhooks. Both are awkward, undocumented patterns with no platform support, and the re-derive-state approach has a real failure mode (a resolved condition can reopen if the re-scan window still contains it).
### Scope: what this touches, and what it does not
**Touches:**
- A new system action, registered the same way `For Each` is (`SYSTEM_ACTIONS` entry in `lib/mcp/workflow-schema-constants.ts`, `lib/step-registry.ts`, codegen templates).
- A new `"waiting"` execution status. `WORKFLOW_EXECUTION_STATUSES`/`NODE_EXECUTION_STATUSES` (`lib/errors/execution-status.ts:39-64`) currently have no waiting/paused state — only pending/running/success/error/skipped/cancelled/phantom/system_error. Needs a migration and run-history UI updates.
- The execution-dispatch routing logic. `resolveDispatchTarget` (`keeperhub-executor/execution-mode.ts:16`) currently picks between `k8s-job`, `in-process`, and `api` based only on `hasWeb3Writes`. The `k8s-job` and `in-process` paths run synchronously in a single pod/process with a hard deadline — `activeDeadlineSeconds` defaults to 300s (`keeperhub-executor/config.ts:99`) — and explicitly no checkpoint/resume (`keeperhub-executor/workflow-runner.ts:264-269`: *"this runner is a standalone K8s-job process with no DevKit run-processor... Tradeoff: no checkpoint/resume"*). Neither can hold a wait longer than the deadline. Only the DevKit durable path, reached via `start()` in `lib/workflow/execute-in-background.ts`, has real durability today. Dispatch routing needs to force any workflow containing a Wait node onto that path.
**Does not touch:** existing node types, the For Each loop mechanism, or any workflow that doesn't use a Wait node — dispatch for those is unaffected.
**Open question that has to be settled before implementation, not discovered mid-PR:** whether the production deployment runs with `WORKFLOW_DISPATCH_VIA_EXECUTOR=1` (`lib/workflow/execute-in-background.ts:80-106`), which bypasses the DevKit `start()` path entirely and routes to SQS/`keeperhub-executor` instead. If so, the durable-sleep primitive may not be reachable in the current production configuration without additional infrastructure work — this needs confirming before the dispatch-routing change is written, since it changes the actual scope of the PR.
### Plan: what you propose
- Add a `Wait` system action with two modes: `duration` (wait N seconds/minutes/hours/days) and `until` (wait until an ISO timestamp), calling `sleep()` from `workflow` inside the step implementation.
- Add a `"waiting"` execution status, surfaced in `workflow_executions`/`workflow_execution_logs` and the run-history UI, distinct from `"running"` (which today implies active compute).
- Change dispatch routing so any workflow whose current node is a Wait node is routed onto the DevKit durable dispatch path, regardless of `hasWeb3Writes`.
- Before writing the dispatch-routing change: confirm via a spike whether prod's `WORKFLOW_DISPATCH_VIA_EXECUTOR` setting affects reachability of the durable path.
- Test plan: kill the pod/process mid-wait and confirm resume; redeploy mid-wait and confirm resume; run an actual multi-day wait against staging, not just a mocked timer.
### Plan: alternatives you considered
- **Polling-based fake wait** (re-enqueue a scheduled check every N minutes until the target time) — rejected. Reinvents durability badly, wastes compute on every poll, and ignores the primitive the platform already depends on.
- **Do nothing, require external cron chaining** — rejected. That's the status quo this issue is about; it's exactly the workaround being described as costly above.
### Scope: compatibility
- [x] Changes database schema or requires a migration.
- [ ] Adds, removes, or upgrades a dependency.
- [ ] Touches authentication, permissions, validation, or spend limits.
- [ ] Changes pricing, plan limits, or anything a user is charged.
Contributor guide
Research direction
Start by checking lib/workflow/execute-in-background.ts and confirming whether WORKFLOW_DISPATCH_VIA_EXECUTOR=1 is used in production. Then read lib/mcp/workflow-schema-constants.ts, lib/step-registry.ts, lib/errors/execution-status.ts, and keeperhub-executor/execution-mode.ts. Done means a Wait node uses durable sleep, survives process termination and redeploys, and passes a real multi-day staging test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, typescript
- Domain
- backend, databases, devops, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100