block / block/buzz

[Bug] buzz-acp reply delivery is implicit — Sonnet-class agents answer in session text and every reply is silently dropped

Open
#2,698 37 comments 6 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

**Describe the bug**

A buzz-acp managed agent's reply reaches the channel only if the model *spontaneously decides* to run `buzz messages send`. Nothing in the injected prompts states that session text is not delivered, and the harness has no fallback when a turn ends without a send — the reply is silently dropped. Strong models happen to infer the convention; weaker ones don't, so **switching an agent's model can silently break all of its replies** while everything looks healthy (turn runs, Activity panel streams the answer, tokens are billed, chat stays empty, exit clean, nothing logged).

**Observed** (Buzz Desktop 0.4.24, Claude Code harness, macOS):

The same agent, same 12,435-char base prompt (byte-identical), same `[Buzz event: @mention]` turn block — only the model differs (via `ANTHROPIC_MODEL`, per the #2692 workaround):

| Session model | Behavior | Chat |
|---|---|---|
| `claude-fable-5` (3 sessions) | ends turn with `Bash: printf '…' \| buzz messages send …` | reply appears ✅ |
| `claude-sonnet-5` (3 sessions) | zero tool calls — answers in plain session text | reply lost ❌ |

The Sonnet reply text is visible in the desktop Activity panel (streamed ACP output) and in the session transcript, so generation succeeded; delivery never happened. The owner sees "working…" then silence — same symptom family as #2459, but here the CLI write doesn't *fail*, it is never attempted.

**Root cause**

The delivery contract is implicit. The base prompt says the `buzz` CLI is "your primary interface" and discusses threading/mentions, but never states "your session text is NOT delivered — reply via `buzz messages send`". The per-turn block (`queue::format_prompt` / `format_event_block`, `crates/buzz-acp/src/queue.rs`) describes the incoming event and names a reply destination without stating the mechanism. Turn completion (`crates/buzz-acp/src/lib.rs`) does not check whether any send occurred for the triggering channel.

**Suggested direction**

1. **State the contract explicitly** in the turn prompt (one line: session text is not delivered; reply with `buzz messages send --channel [--reply-to ]`). Cheap, model-agnostic, immediately effective — verified locally: adding the equivalent instruction to the agent's system prompt makes Sonnet 5 post correctly.
2. **Add a harness fallback**: when a prompted turn completes and no `buzz messages send` targeting the triggering channel was observed, either auto-post the final ACP text as the reply (best) or at minimum log at WARN and surface a delivery-failure indicator to the owner. This also covers #2459's failed-write case — silent loss is the real bug in both.

**To Reproduce**

1. Managed Claude Code agent; set `ANTHROPIC_MODEL=claude-sonnet-5` in its env vars (needed because of #2692); restart.
2. DM the agent any question.
3. Activity panel streams a correct answer; no message ever arrives in the chat; harness log shows nothing between turn start and the next lifecycle event.

**Related**: #2459 (reply lost when CLI write fails), #2265 (silent failure when default model lacks credits), #2692 (model setting not applied — the fix that exposes this one, since it makes cheaper models reachable).

**Environment**: Buzz Desktop 0.4.24 (Block-signed), `@agentclientprotocol/claude-agent-acp` 0.60.0, macOS Intel, relay `communities.buzz.xyz`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.