buzz-acp: `subscribe=all` agents wake on each other's 👀/💬 bookkeeping reactions, producing a self-sustaining reaction storm
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
### Summary
Two or more agents owned by the same human, both on `BUZZ_ACP_SUBSCRIBE=all`, will storm each
other indefinitely without a single chat message being sent. Each agent wakes on the *other's*
👀/💬 pickup reactions, which are harness bookkeeping rather than user intent.
This is a sibling of #3649 but it is **not** fixed by #4040. That PR guards ephemeral kinds
(20000-29999). Reactions are kind **7** and their cleanup deletions are kind **5**: regular,
stored, and still delivered to a wildcard subscriber. On our relay the reaction loop was roughly
**20x larger** than the typing-indicator burn that #3649 describes, so `subscribe=all` will look
fixed after #4040 while this remains open.
It also does **not** require `respond_to=anyone`. See "Why default settings are enough" below.
### Mechanism
Every event an agent picks up generates four more events of its own:
| step | source | kind |
|---|---|---|
| `reaction_add(eid, "👀")` at queue-push | `lib.rs:2212` | 7 |
| `reaction_add(eid, "💬")` before the prompt fires | `pool.rs:1881` | 7 |
| `ReactionGuard` removes 👀 on turn end | `pool.rs:3380` | 5 |
| `ReactionGuard` removes 💬 on turn end | `pool.rs:3380` | 5 |
All four are published into the same channel. With `subscribe=all` and no `BUZZ_ACP_KINDS`
override, the subscription is a wildcard in two independent places:
- `config.rs` `resolve_channel_filters` / `resolve_dynamic_channel_filter`: `SubscribeMode::All`
yields `ChannelFilter { kinds: config.kinds_override.clone(), require_mention: false }`, which
is `None` when unset;
- `relay.rs` `send_subscribe`: `kinds` is omitted from the `REQ` entirely when the filter is
`None`, so the relay pushes every kind in the channel;
- `lib.rs` rules construction: `SubscribeMode::All` yields
`SubscriptionRule { kinds: kinds_override.unwrap_or_default(), .. }` = an empty vec, and
`filter.rs` `match_event` documents empty as wildcard
(`if !rule.kinds.is_empty() && !rule.kinds.contains(..)`).
So with N wildcard agents in a channel, one real message produces 4N bookkeeping events, each of
which wakes the other N-1 agents, each of those producing 4 more. `ignore_self` does not help:
it only suppresses an agent reacting to its own events, and A -> B -> A is exactly the case it
misses. There is no reply-depth counter, hop limit, or cooldown anywhere in the path.
### Why default settings are enough to trigger it
`respond_to=owner-only` is the default and does **not** prevent this. The inbound author gate in
`lib.rs` admits same-owner siblings in both `OwnerOnly` and `Allowlist` modes, by design:
> Both OwnerOnly and Allowlist accept events from "siblings" - pubkeys whose
> `agent_owner_pubkey` matches this agent's owner (e.g. other bots launched by the same human).
So any human running two agents on `subscribe=all` hits this on stock config. Given #3204 /
#3277 / #2508 make `subscribe=all` the standard workaround for reaching an agent on another
machine, that combination is not exotic.
### Measured impact
Self-hosted relay (compose stack from `deploy/compose`), buzz pinned at `d40a3329`, three agents
owned by one human: `buzz-agent` 0.1.0, OpenCode 1.18.10, and `claude-agent-acp` 0.64.0. All
three were on `subscribe=all` with no `BUZZ_ACP_KINDS`.
Event counts, straight from the relay's postgres:
| metric | count |
|---|---|
| kind 7 (reactions) stored | **25,717** |
| kind 9 (actual chat messages) stored | **304** |
| reactions whose `e` target is another **kind 7** | **22,373** |
| reactions whose `e` target is a **kind 5** deletion | 2,967 |
| reactions whose `e` target is a real **kind 9** message | 338 |
So 87% of all reactions were reactions to reactions. Per author, all three agents, none of it
human:
| author | 👀 | 💬 |
|---|---|---|
| agent A | 8,971 | 3,210 |
| agent B | 8,586 | 2,982 |
| agent C | 1,010 | 954 |
| the human owner | 1 | 2 |
Peak was 20,113 kind-5/7 events in a single hour. Agent-side symptoms: 5,650 HTTP 429s in 30
minutes on one agent, 729 `pool_exhausted` on another, and the 500-message queue cap hit.
Incidentally confirming #3649 from the same deploy: the relay rejected **1,063** reaction
publishes with `invalid: reaction target event not found`, which is the fingerprint of an agent
trying to 👀 an ephemeral typing/presence event that was never stored.
### Reproduction
1. Two agents, same owner, both members of one channel.
2. Both on `BUZZ_ACP_SUBSCRIBE=all` with `BUZZ_ACP_KINDS` unset. `respond_to` can stay at its
`owner-only` default.
3. Post one message in the channel.
4. Both agents 👀 it, then 👀 each other's 👀, and it does not converge.
### Suggested fix
Three options, roughly in order of how well they preserve intent:
1. **Mark harness-generated reactions.** 👀/💬 are bookkeeping, not user intent, but are
indistinguishable from a human reaction on the wire. A tag on the reaction event would let
`match_event` drop them while still waking on genuine human reactions.
2. **Skip reactions authored by sibling agents.** Narrower, no wire format change, but it also
suppresses deliberate agent-to-agent reactions.
3. **Give `SubscribeMode::All` a curated default kind list** when `kinds_override` is `None`,
the same shape mentions mode already has.
On option 3, one correction to the suggestion in #3649: that issue proposes a default allowlist
"mirroring the mentions-mode one **plus reactions and canvas**", and the workaround it recommends
is `BUZZ_ACP_KINDS=9,40002,40003,5,7,1059,40100,40007,46010,46020`. Including `5` and `7` is
exactly what detonates in a multi-agent channel. That allowlist is safe for a single-agent
deploy, which is presumably where it was validated, but it should carry a warning or drop those
two kinds.
### Workaround
Same allowlist, minus the deletions and reactions:
```
BUZZ_ACP_KINDS=9,40002,40003,1059,40100,40007,46010,46020
```
Applied to the one agent we keep on `subscribe=all`; the rest run `subscribe=mentions`, which is
already narrow by construction. This took the relay from 20,113 bookkeeping events an hour to
zero reaction-on-reaction traffic. Note this is a strict superset of what #4040 fixes, so it
remains necessary after that PR lands.
Contributor guide
Research direction
Start with the reaction paths in lib.rs:2212, pool.rs:1881 and pool.rs:3380, then trace SubscribeMode::All through config.rs, relay.rs and filter.rs. Compare the three suggested mitigation options and define a regression scenario from the two-agent reproduction; done should prevent bookkeeping reactions from sustaining the loop without losing the intended subscription behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100