Renewable CPU-rate budget ('CPU per minute') for long-running guest executions (agents)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 4.6k
- Forks
- 251
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 34
Description
Problem
Guest execution limits that are totals or wall-clock based are the wrong
shape for long-running, I/O-bound jobs — most importantly agent sessions.
An agent session (e.g. the pi ACP adapter) is a long-lived orchestrator that
runs for minutes across many LLM turns, spending ~95% of that time waiting
on the LLM, the network, and tool I/O — using almost no CPU. A per-execution
wall-clock limit counts that wait time, so a perfectly well-behaved agent is
killed mid-session:
Error: Script execution exceeded the wall-clock limit (limits.jsRuntime.wallClockLimitMs)
ACP adapter process ... exited with code 1 ... live session route evicted
(Observed in the agent load test: with realistic 1.5–6 s LLM latency × N tool
calls, a session runs past the limit and the adapter is terminated.)
Wall-clock conflates three distinct failure modes into one blunt cap:
- using too much CPU (a runaway/spinning execution),
- hung (blocked forever, using ~0 CPU),
- running too long (a legitimately huge session).
For a short untrusted tool call collapsing all three into "kill at 20 s" is
fine. For a long-lived agent runtime it is wrong — you must separate them.
Proposed fix: renewable CPU-rate budget ("CPU per minute")
Bound the long-lived execution by a renewable CPU-rate budget — the Linux
cgroup cpu.max model (quota per period, refilling), i.e. "X CPU-seconds per
1 s window":
- An I/O-bound agent that is mostly idle-on-CPU stays under budget indefinitely
→ runs the whole session, no guillotine. - A runaway/spinning execution pegs a core → burns its per-window allowance →
throttled/killed. This is what wall-clock was trying to catch, done correctly. - Rate-based ⇒ works for arbitrarily long sessions without a magic "max minutes".
Complements:
- Idle / no-progress timeout — a CPU-rate budget can't see a process stuck on
0 CPU (case 2 above), so kill on no LLM/tool activity for N seconds. - Optional session-max ceiling (generous, e.g. 30 min) as a final backstop.
Implementation note: the machinery already exists — the V8 CpuBudgetGuard
(the interrupt that enforces cpuTimeLimitMs today) is a total budget. Making
it renewable (refill over wall-clock time) turns it into exactly this rate
limiter, with no new subsystem and without per-thread cgroups (which
docs/design/unified-sidecar-runtime.md says to avoid for isolates).
Other caps with the same "total, should be renewable" flaw
While here, these per-execution caps are also totals/wall-clock-based and bite
long-running jobs the same way — they should move to the renewable model:
limits.jsRuntime.cpuTimeLimitMs— total accumulated CPU per execution
(defaultDEFAULT_V8_CPU_TIME_LIMIT_MS = 30_000). A long agent doing real work
accumulates >30 s CPU over minutes and is killed. This is the primary target
(wall-clock already defaults to0/disabled; CPU-time does not).limits.resources.maxWasmFuel— WASM fuel, enforced as a wall-clock
timeout for the WASM runtime (crates/execution/src/wasm.rs). Same total/
wall-clock shape; a long-running WASM command should get a renewable fuel rate.
Instantaneous caps (maxProcesses, maxOpenFds, maxSockets, v8HeapLimitMb,
maxWasmMemoryBytes, pendingEvent*, maxFilesystemBytes, maxInodeCount) are
"how many at once / how much stored" — not consumption budgets — and correctly
stay as-is.
Interim state
- Runtime default wall-clock is already disabled (
DEFAULT_V8_WALL_CLOCK_LIMIT_MS = 0). - The load-test actor (
packages/load-tests/src/compute/server.ts) had an explicit
wallClockLimitMs: 20_000that killed agent sessions; it has been set to0
(disabled) with a TODO pointing here. cpuTimeLimitMsis left in place for now; once the renewable budget lands, both
wall-clock and CPU-time should re-enable a sane renewable default.
Acceptance
- Renewable CPU-rate budget (
cpu.max-style) for guest executions, config-exposed. - Idle/no-progress timeout for the hung case.
-
cpuTimeLimitMsandmaxWasmFuelmigrated to (or complemented by) the rate model. - A long (minutes) mostly-idle agent session runs to completion; a CPU spinner
is still throttled/killed; a hung execution is still reaped. - Sane renewable defaults re-enabled (replacing the disabled wall-clock).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the V8 CpuBudgetGuard and the execution limits described in the issue, then inspect docs/design/unified-sidecar-runtime.md and crates/execution/src/wasm.rs. Review packages/load-tests/src/compute/server.ts for the agent-session scenario. Done means renewable CPU limits, idle/no-progress handling, migrated CPU/WASM caps, and tests showing idle sessions complete while spinners and hung executions are stopped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, linux, rust, wasm
- Domain
- backend, infrastructure, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100