HarperFast / HarperFast/harper

Workflow developer API surface (Workflow base class, ctx.step, ctx.atomic, ctx.signal, ctx.sleep)

Open
#757 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Part of #752 (Durable Execution v0.1 epic).

## Goal

A developer-facing API for declaring durable workflows as an extension of the existing Resource/Component model — not a separate system bolted on. A Workflow should be declared as a class in a Harper component, registered like any other Resource, invokable over REST/MQTT/WebSocket/SSE, and observable through Harper's existing audit and analytics surfaces.

## API shape

\`\`\`ts
import { Workflow, tables } from 'harper';

export class TriageSupportTicket extends Workflow {
static input = { ticketId: 'string' };

async run(ctx) {
const ticket = await ctx.step('loadTicket', () =>
tables.SupportTicket.get(ctx.input.ticketId)
);

const proposal = await ctx.step('draftResponse', (memo) =>
llm.chat({ idempotencyKey: memo.idempotencyKey, messages: [...] })
);

let response = proposal.response;
if (proposal.confidence < 0.8) {
const review = await ctx.signal('humanReview', { timeout: '24h' });
response = review.approvedResponse;
}

await ctx.atomic('resolve', (txn) => {
txn.tables.SupportTicket.patch(ticket.id, { status: 'resolved', response });
txn.tables.ResolvedTicket.put({ id: ticket.id, embedding, resolution: response });
txn.tables.AuditEntry.put({ event: 'ticket_resolved', workflowId: ctx.id });
});
}
}
\`\`\`

## Primitives

| API | Semantics |
|---|---|
| \`ctx.step(name, fn)\` | General checkpoint boundary. Body can do anything (Harper read, LLM call, HTTP). Returns checkpointed result on replay. At-least-once execution of body + result memoization. |
| \`ctx.atomic(name, (txn) => …)\` | Workflow checkpoint commits *inside* a Harper \`transaction(fn)\` with the user's table writes. Exactly-once for the in-Harper portion. **The differentiator surface.** |
| \`ctx.signal(name, opts?)\` | Wait for an external signal — human approval, callback, message arrival. Survives node restarts; resumes on signal arrival or timeout. |
| \`ctx.sleep(duration)\` | Durable sleep. Rides the timer service (#754) and survives restarts. |
| \`ctx.condition(predicate)\` | Wait until a predicate over workflow state evaluates true. (Lower priority for v0.1 — can be derived from signal + sleep.) |
| \`ctx.id\`, \`ctx.input\`, \`ctx.attempt\`, \`ctx.startedAt\` | Identity / metadata. |

## Design decisions worth flagging

**\`ctx.atomic\` vs \`ctx.step\` — why both.** \`ctx.step\` is general: body may be an LLM call, HTTP request, vector search, anything. Checkpoint is a separate commit because the body may have written to external systems that can't ride inside a Harper transaction. \`ctx.atomic\` is narrow and powerful: body only writes to Harper tables via the supplied \`txn\` handle, and the checkpoint commits *with* those writes in one LMDB/RocksDB transaction. Making every step atomic is neither correct (HTTP calls don't roll back) nor cheap (would pay write-txn overhead per step).

**Steps are explicit, not implicit.** Every \`await ctx.step(...)\` is a checkpoint boundary. The alternative — auto-checkpointing every \`await\` — is what LangGraph's durability mode does, but it introduces determinism constraints on user code that are hard to debug. Explicit steps trade slightly more verbose code for a much clearer mental model.

## Invocation surfaces

\`\`\`ts
const handle = await workflows.TriageSupportTicket.start({ ticketId: 't-7421' });
await workflows.signal(handle.id, 'humanReview', { approvedResponse: '...' });
const state = await workflows.get(handle.id);
await handle.subscribe((event) => { /* lifecycle stream */ });
\`\`\`

External clients reach the same primitives over REST/MQTT/WebSocket/SSE via the existing Resource framing.

## Where the work lives

Mostly userland — \`Workflow\` base class, the \`ctx\` shape, the invocation API, observability integration. **Two core-adjacent paths:**

1. **\`ctx.atomic\` splice into \`transaction(fn)\`.** The workflow checkpoint write needs to enroll on the same \`RocksTransaction\` / \`LmdbTransaction\` the user's writes are committing on. Best built in close coordination with [\`resources/transaction.ts\`](https://github.com/HarperFast/harper/blob/main/resources/transaction.ts) and [\`resources/DatabaseTransaction.ts\`](https://github.com/HarperFast/harper/blob/main/resources/DatabaseTransaction.ts).
2. **\`ctx.step\` interception fast path.** At hot-loop scale (tool-use, fan-out/fan-in), the per-step runtime decision (replay-cached output vs. execute body, idempotency-key generation, checkpoint commit) is on the critical path. Lives in the workflow runtime, not userland.

Everything else — \`Workflow\` discovery, REST/MQTT wiring, the \`handle.subscribe\` lifecycle stream — can ride existing Resource/Component machinery.

## Observability

Workflow events should flow through Harper's existing analytics pipeline so operators see workflow turn / step / cost telemetry alongside Resource handler metrics, not in a separate UI.

## Dependencies

- Step memoization (#755) — for the checkpoint table.
- Determinism contract (#756) — for the runtime that intercepts \`ctx.step\` and shadows non-deterministic globals.
- Timer service (#754) — for \`ctx.sleep\`.
- Consensus lane (#753) — for checkpoint durability.

## Open questions

- Declarative graph DSL (LangGraph-style) alongside imperative TS? Likely v0.2.
- Resource subscription path for \`handle.subscribe\` vs. a workflow-specific signal primitive? Proposal flags this as the POC-tier item that will likely want optimization later.

---

🤖 Filed by [Claude](https://claude.com/claude-code) on behalf of @kriszyp

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.