buzz-acp: online mentions can be silently lost before enqueue and in native-steer withholding
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Summary
On Buzz Desktop / `buzz-acp` v0.5.4 on Windows, correctly `p`-tagged mentions to an online managed agent can be accepted and persisted by the relay but never reach the agent queue or session. We observed two distinct silent-loss seams in one incident:
1. A live-subscription/pre-queue gap: the harness was connected and subscribed before the event was published, but the event produced no queue acceptance, lazy-pool wake, or session.
2. A per-channel native-steer/withhold gap: events published during an active turn never appeared in the ACP session and had no steer-ack receipt; a later event after the visible turn also spawned nothing. The state was volatile and a later process replacement discarded it.
This is a silent-success failure: the relay accepts the event and the agent can appear online/ready, but the mention disappears without a sender- or operator-visible error.
## Sanitized incident timeline (UTC, 2026-08-04)
- 15:56:54: harness connected to a hosted private relay.
- 15:57:00: subscribed to the DM channel.
- 15:58:34: event `7d3dda2b…` persisted with the correct recipient `p` tag. No pool wake, queue receipt, or session followed.
- 16:20:57: a fresh process reported `agent_pool_ready agents=10`.
- 16:21:30: a session started from an earlier catch-up event.
- 16:22:22 and 16:23:42: events `2bd93407…` and `e5293983…` persisted with the correct `p` tag while the turn was active. A literal scan of the session transcript found zero occurrences of both IDs, and no steer-ack was logged.
- 16:24:12: the visible turn ended.
- 16:27:52: event `f5aed6c9…` persisted with the correct tag but spawned no follow-on session.
- 16:29:21: the managed process was replaced, discarding any volatile queued/withheld state.
- A fresh subscription later recovered an event that already existed on the relay before subscribe, showing that catch-up could work even while live delivery was unreliable.
## Relevant v0.5.4 paths
- [Accepted-event queue and dispatch path](https://github.com/block/buzz/blob/desktop-v0.5.4/crates/buzz-acp/src/lib.rs#L2070-L2137)
- [Lazy-pool wake path](https://github.com/block/buzz/blob/desktop-v0.5.4/crates/buzz-acp/src/lib.rs#L1652-L1686)
- [Mid-turn native-steer fork](https://github.com/block/buzz/blob/desktop-v0.5.4/crates/buzz-acp/src/lib.rs#L2116-L2169)
- [Lost-ack leak warning](https://github.com/block/buzz/blob/desktop-v0.5.4/crates/buzz-acp/src/lib.rs#L1571-L1581)
- [Steer-ack release/drop handler](https://github.com/block/buzz/blob/desktop-v0.5.4/crates/buzz-acp/src/lib.rs#L2473-L2508)
- [Synchronous withhold before prompt completion](https://github.com/block/buzz/blob/desktop-v0.5.4/crates/buzz-acp/src/lib.rs#L2807-L2839)
The source itself warns that losing the steer acknowledgement can leak a withheld event until the long in-flight deadline.
## Evidence boundary
Current INFO logs do not record event IDs at receive, filter, enqueue, withhold, ack, dequeue, or spawn boundaries. Therefore:
- The first event is narrowed to live relay delivery vs. an unlogged pre-queue filter branch; it definitively never reached session spawn.
- The mid-turn events are code-supported narrowing to the native-steer/withheld-event seam, not proof of the exact internal branch.
## Expected behavior
- Every accepted matching event is durably queued or replayed after reconnect/restart.
- A lost steer acknowledgement is bounded by a short timeout and releases the withheld event to normal dispatch.
- “Online/ready” reflects working relay ingress, not only process liveness.
- Operators can trace an event ID through receive → filter → enqueue → withhold → ack → dequeue → session.
## Suggested fixes
1. Persist a per-channel high-water mark and accepted/withheld queue.
2. Catch up from that mark on reconnect and periodically; dedupe by event ID.
3. Bound steer-ack waits to seconds and atomically release unacknowledged events on timeout or prompt completion.
4. Persist queue state across harness replacement.
5. Add event-ID transition logs and an end-to-end external canary.
Related: #4743 covers hosted-relay fan-out / silently dead subscriptions; #4189 covers a different native-steer deadlock for non-steerable ACP agents.
Contributor guide
Assessment
This issue has not been assessed yet.