HarperFast / HarperFast/harper

Native durable timer service (sharded hierarchical timing wheel)

Open
#754 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

Part of HarperFast/harper#752 (Durable Execution v0.1 epic).

## Problem

Harper has no general user-facing scheduler today.

- The \"jobs\" subsystem in [\`server/jobs/jobRunner.ts\`](https://github.com/HarperFast/harper/blob/main/server/jobs/jobRunner.ts) is a fixed-enum dispatcher over a dozen bulk operations (\`csv_file_load\`, \`import_from_s3\`, \`delete_records_before\`, …) — no cron, no delay primitive, no retry policy.
- The only user-facing periodic mechanism is record-TTL eviction ([\`scheduleCleanup\` in \`Table.ts:4458\`](https://github.com/HarperFast/harper/blob/main/resources/Table.ts#L4458)), a self-rearming \`setTimeout\`, not a queue.
- Internal \`setInterval\`s exist for analytics flush, throttling, read-txn GC — none are user-schedulable.

Durable execution needs \`ctx.sleep\`, retry-with-backoff, scheduled signals, cron — all of which require a real timer service. At workflow scale, polling a table of \`(fireAt, …)\` rows is fine for a POC but will not be benchmark-competitive when millions of timers are pending (typical when many workflows have long sleeps).

## Proposed shape

A **sharded hierarchical timing wheel implemented in core**, replicated through the consensus lane (#753).

- **Hierarchical wheels**: multiple wheels at coarser granularities (ms / sec / min / hr / day) so a 30-day sleep doesn't occupy any per-ms slot — it lives in the coarsest wheel and cascades down as time approaches.
- **Sharding**: partitioned by workflow ID, matching the consensus sharding. Each shard runs its own wheel and advances independently — removes contention and lets the firing work scale horizontally.
- **Properties**: O(1) insertion, O(1) tick advance, bounded-latency fire selection independent of the number of pending timers.
- **Durability**: timer state replicates via the consensus lane so a fire is not lost on node failure; on takeover, the new leader resumes the wheel at the correct logical time.

## Is this load balancing?

The primary motivation is algorithmic efficiency, not load balancing. Distribution across nodes is handled at the shard-to-node assignment layer, not inside the wheel itself. That said, the timing-wheel implementation should emit observable signals (timers fired/sec per shard, fire-latency histogram, queue depth per granularity) so a higher-layer rebalancer can act on hot-shard signals later. No closed-loop rebalancing in v0.1; just instrumentation.

## Coordination with HarperFast/harper-pro#676

[#626 (Built-in Harper Agent Component)](https://github.com/HarperFast/harper-pro/issues/676) defines a \`schedule_followup({ delayMs, prompt })\` tool implemented with \`setTimeout(...).unref()\` on the main thread. That works for the immediate scope of HarperFast/harper-pro#676 (autonomous follow-up while the main thread stays up) but does not survive a restart — exactly the scenario the durable timer service is for. Once this service lands, \`schedule_followup\` should be migrated to use it. No need to block HarperFast/harper-pro#676 on this.

## Open design questions

- Wheel widths and number of levels — likely 256 × 4–5 levels covering ms → days; concrete sizing depends on target workload.
- Sub-millisecond timers — useful or out of scope? (Probably out of scope; existing \`setTimeout\` is fine for that regime.)
- Cron expression support in v0.1 or v0.2? Cron sits *above* the wheel as a rule-driven re-insertion policy; lightweight to add if the wheel is clean.
- Cancellation cost — must be O(1) too, which means we need a stable slot-handle per scheduled timer.

## Out of scope

Not a general application-level cron-like scheduler for arbitrary user code in v0.1; v0.1 only powers the workflow runtime. General-purpose user scheduling can layer on later.

---

🤖 Filed by [Claude](https://claude.com/claude-code) on behalf of @kriszyp

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.