ACP harness: agents silently fail to publish responses (no safety net for missed buzz messages send)
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Problem
Agents running under the ACP harness sometimes produce complete, correct responses but **fail to publish them** to the Nostr relay — the response exists in local stdout but never reaches the channel. Other users see the agent as unresponsive or "thinking forever."
### Root cause
The current architecture relies **solely on the system prompt** (`base_prompt.md`) to instruct agents to call `buzz messages send`. There is no harness-level safety net. When an LLM agent, deep in a complex multi-step reasoning chain, overlooks the final publish step, the entire turn's output is silently lost.
```
User message → relay → ACP harness → session/prompt → Agent runs
↓
Generates response
(reasoning + tools + text)
↓
❌ Forgets buzz messages send
↓
Response lost — nothing on relay
```
This is a **structural gap**, not a per-agent bug. Every agent type (Claude Code, Codex, Grok, OpenCode) is susceptible.
### Proposed fix (three layers)
| Layer | Scope | Location | Effort |
|:------|:------|:---------|:-------|
| **L1** | Single agent | Agent core memory (`buzz mem set core`) | Immediate, one command |
| **L2** | All agents | `base_prompt.md` — "first and last" reinforcement | Already implemented (see below) |
| **L3** | Platform (root fix) | `buzz-acp` harness — post-turn publish check | ~50-100 lines Rust |
### L2: base_prompt.md changes (already implemented locally)
Two additions to `crates/buzz-acp/src/base_prompt.md`:
**1. Top of file** — A CRITICAL banner right after the intro, before any other content:
```markdown
**CRITICAL — Read this first:** Your reasoning and tool calls are invisible to
other users. The ONLY way anyone sees your response is if you call
`buzz messages send`. A result, answer, or deliverable that you generate but
don't publish does not exist. Every turn that produces something worth
communicating MUST end with `buzz messages send`. No exceptions.
```
**2. End of file** — A "Before Ending Your Turn" sentinel with checklist:
```markdown
## Before Ending Your Turn
**STOP. Ask yourself:** Did I produce a result, answer, deliverable, decision,
or blocker that someone needs to see?
If yes — **you MUST call `buzz messages send` before ending this turn.**
...
Checklist:
1. Was the user waiting on an answer? → buzz messages send
2. Did you finish delegated work? → buzz messages send (with callback @mention)
3. Did you find something worth reporting? → buzz messages send
4. Are you blocked and need input? → buzz messages send
5. Nothing to communicate? → Silence is correct. End without sending.
```
The design intent is "first and last" — the agent sees the CRITICAL banner immediately on session start, and is intercepted by the checklist before ending a turn.
### L3: Harness-level safety net (proposed design, not yet implemented)
The ACP harness already knows the target channel UUID (it's in the session context). After `session/prompt` returns with `EndTurn`, the harness should:
1. Query the relay: did the agent publish any events to the target channel since the turn started?
2. If no events were published AND the agent produced stdout output → log a warning + optionally auto-publish a fallback message
3. This catches the "agent produced output but forgot to publish" case regardless of agent type
Pseudo-code location: around `crates/buzz-acp/src/acp.rs` `session_prompt_with_idle_timeout` return path, or in the pool dispatch layer in `pool.rs`.
### Impact assessment
- **Without this fix**: Silent failures erode trust in agent reliability across the platform
- **L2 alone**: Reduces but doesn't eliminate the failure mode (prompt-based, not enforced)
- **L2 + L3**: Prompt reinforcement + infrastructure guarantee = defense in depth
Contributor guide
Research direction
Read crates/buzz-acp/src/acp.rs, especially session_prompt_with_idle_timeout, and then inspect the pool dispatch layer in pool.rs. Trace how the harness knows the target channel and how stdout and relay events are handled after EndTurn. Done means detecting a turn with stdout but no published event, logging a warning, and determining whether a fallback publish is appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- devtools
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100