KeeperHub / KeeperHub/keeperhub
Alternative workflow runtime (Rust or Bun) conforming to the Workflow DevKit protocol
- Dominant language
- TypeScript
- Stars
- 24
- Forks
- 93
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 253
Description
Part of #2294.
## Reason
The workflow engine runs on the Vercel Workflow DevKit: `workflow` 4.8.5 and `@workflow/world-postgres` 4.3.5 are direct dependencies, durable state lives in the Postgres world (`pnpm db:setup-workflow` runs `@workflow/world-postgres/bin/setup.js`), and the per-execution runner is `keeperhub-executor/workflow-runner.ts`, started under `tsx` inside a Job container.
We are open to a different runtime underneath that - a Rust implementation, or Bun - provided it speaks the same protocol and the same Postgres world, so this is a runtime swap rather than a fork.
**Read the other sub-issues on the tracking issue before starting.** The two costs we can currently point at (a mean 5s fixed jitter, and a cold Kubernetes Job per execution) sit outside the runtime, so a faster runtime is upside on top of those rather than a substitute for them. That is a sequencing note, not a rejection: this issue is open and the work is wanted.
## Scope
An alternative runtime for workflow execution that conforms to the Workflow DevKit protocol and the `@workflow/world-postgres` durable-state format.
Two shapes are in scope, and they are very different sizes:
- **Bun.** Smallest delta. The engine and the plugin steps stay TypeScript; the question is measured gain and whether everything survives - the `node:vm`-based sandbox path, `postgres-js`, and the `tsx` bootstrap in particular.
- **Rust.** Largest delta, and the interesting one. See the crux below.
## The crux for a Rust implementation
Every plugin step in this repository is TypeScript (`plugins/`), and there are a lot of them. A Rust runtime therefore has to do one of:
1. Drive the existing TypeScript steps, embedding a JS engine and paying its cost at the boundary
2. Reimplement steps in Rust, which is a rewrite of the plugin surface rather than a runtime change
3. Handle only the durable-execution and orchestration layer in Rust, leaving step bodies where they are
State which of the three you are proposing in your first comment. They have different costs, different risks, and only the third is plausibly incremental.
## What a contribution needs to show
- **Protocol conformance, demonstrated not asserted.** The same workflow definition must produce the same durable state transitions in the Postgres world, and existing execution rows written by the current runtime must remain readable. A runtime that cannot resume an execution the current one started is a fork.
- **Measured before and after.** A profiling framework already exists at `scripts/runtime/workflow_runtime_analysis/` (static analysis, V8 step profiling, whole-workflow profiling, WASM fuel calibration). Use it rather than building a new one, and note its own recorded finding that static complexity does not predict runtime cost.
- **Which surface is replaced**, stated explicitly: step execution only, orchestration only, or the whole engine.
- **The sandbox story.** User JavaScript from the Code node runs in a separate hardened service (`sandbox/`). Say whether your runtime changes that boundary. It must not weaken it.
- **A migration path.** Both runtimes will run concurrently during any rollout, against one Postgres world.
## Note on the ordering
If #2289 has landed, use its numbers to state what fraction of end-to-end latency your change actually addresses. If it has not, measure the runner stage locally and show the figure. A runtime change justified by interpreter benchmarks rather than by workflow execution timings will be hard to review.
---
Tracking: [KEEP-1271](https://linear.app/keeperhubapp/issue/KEEP-1271) (internal tracker, not publicly accessible)
Contributor guide
Research direction
Read the related sub-issues first, then inspect keeperhub-executor/workflow-runner.ts, plugins/, sandbox/, and scripts/runtime/workflow_runtime_analysis/. Choose and state one Rust approach or evaluate Bun, measure the runner and workflow timings, and compare protocol state transitions and resume behavior against the existing Postgres world. Done means demonstrated conformance, preserved sandbox isolation, explicit replacement scope, and a concurrent-runtime migration path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bun, kubernetes, node.js, postgresql, rust, typescript
- Domain
- backend, databases, devops, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100