anomalyco / anomalyco/opencode
core: bound server-side interrupt settlement latency
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 209k
- Forks
- 27.5k
- PR merge metrics
- PR metrics pending
Description
Problem
Interrupting a running turn can take several seconds before the server settles the execution and projects its terminal state. In one observed run, shell/tool cancellation was timestamped at 16:33:49.400Z, while the assistant terminal failure was not created until 16:34:00.946Z and became visible around 16:34:01Z.
This is actual server-side settlement latency, not just delayed client feedback.
Observed behavior
- The user interrupts a turn with an active shell/provider step.
- Tool cancellation begins promptly.
- The interrupt request remains pending while cleanup and durable publication settle.
- The execution does not reach its durable interrupted terminal for roughly 11.5 seconds.
Several unrelated sessions resumed logging at approximately the same time, suggesting shared service contention rather than only per-session shell teardown.
Current evidence
Two independent blocking paths are confirmed in the runner:
- Interrupted settlement waits for
captureStepEnd(). A blocked end snapshot prevents both the run and interrupt request from settling. - Durable events across sessions share a single SQLite transaction permit. An unrelated open transaction can prevent the interrupted step from durably settling.
The historical timestamp pattern points more strongly at durable-event/SQLite contention than end-snapshot work: event timestamps are assigned before durable publication waits, and unrelated sessions showed synchronized progress after the stall. This is not conclusive because phase-level timing was absent.
Other unbounded cleanup paths remain relevant, including shell close waiting and provider stream cancellation.
Desired outcome
Interrupt settlement should have bounded latency under snapshot, database, shell, and provider contention without weakening:
- durable event ordering
- exact changed-file attribution
- Undo/revert behavior for partial file mutations
- single terminal-event semantics
Recommended investigation
- Add phase timing around tool terminal publication, tool-fiber joins, end-snapshot capture, changed-file calculation, step terminal publication, execution terminal publication, and shell/provider finalizers.
- Record SQLite transaction-permit queue time separately from transaction execution and listener time.
- Reproduce the long-tail interrupt with instrumentation.
- Fix the measured blocker rather than masking it in clients.
- Add a latency regression test using deterministic gates, not wall-clock sleeps.
Potential directions to evaluate after measurement:
- Replace native SQLite busy waiting with cooperative, bounded retry so the Bun event loop remains schedulable.
- Reduce or isolate work performed while holding the global durable transaction permit.
- Bound interrupted end-snapshot work while preserving the exact pre/post boundary needed by Undo.
- Bound shell and provider cancellation finalizers with explicit escalation and observable failure states.
Test seams
The exploration branch interrupt-latency-exploration contains deterministic characterization tests in packages/core/test/session-runner.test.ts for:
- blocked interrupted end-snapshot capture
- unrelated database transaction contention during interrupted settlement
The client-side Stopping... prototype from #42115 was intentionally closed because it improved perceived responsiveness but did not reduce server latency.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the deterministic characterization tests in packages/core/test/session-runner.test.ts on the interrupt-latency-exploration branch, covering blocked end-snapshot capture and SQLite transaction contention. Add phase and transaction-permit timing, then reproduce the long-tail interrupt to identify the blocker. Done means bounded settlement latency with deterministic regression coverage while preserving event ordering, changed-file attribution, Undo behavior, and single terminal events.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sqlite, typescript
- Domain
- backend, databases, performance, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100