block / block/buzz

ACP harness: agents silently fail to publish responses (no safety net for missed buzz messages send)

Open
#4,709 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

## Problem

Agents running under the ACP harness sometimes produce complete, correct responses but **fail to publish them** to the Nostr relay — the response exists in local stdout but never reaches the channel. Other users see the agent as unresponsive or "thinking forever."

### Root cause

The current architecture relies **solely on the system prompt** (`base_prompt.md`) to instruct agents to call `buzz messages send`. There is no harness-level safety net. When an LLM agent, deep in a complex multi-step reasoning chain, overlooks the final publish step, the entire turn's output is silently lost.

```
User message → relay → ACP harness → session/prompt → Agent runs

Generates response
(reasoning + tools + text)

❌ Forgets buzz messages send

Response lost — nothing on relay
```

This is a **structural gap**, not a per-agent bug. Every agent type (Claude Code, Codex, Grok, OpenCode) is susceptible.

### Proposed fix (three layers)

| Layer | Scope | Location | Effort |
|:------|:------|:---------|:-------|
| **L1** | Single agent | Agent core memory (`buzz mem set core`) | Immediate, one command |
| **L2** | All agents | `base_prompt.md` — "first and last" reinforcement | Already implemented (see below) |
| **L3** | Platform (root fix) | `buzz-acp` harness — post-turn publish check | ~50-100 lines Rust |

### L2: base_prompt.md changes (already implemented locally)

Two additions to `crates/buzz-acp/src/base_prompt.md`:

**1. Top of file** — A CRITICAL banner right after the intro, before any other content:
```markdown
**CRITICAL — Read this first:** Your reasoning and tool calls are invisible to
other users. The ONLY way anyone sees your response is if you call
`buzz messages send`. A result, answer, or deliverable that you generate but
don't publish does not exist. Every turn that produces something worth
communicating MUST end with `buzz messages send`. No exceptions.
```

**2. End of file** — A "Before Ending Your Turn" sentinel with checklist:
```markdown
## Before Ending Your Turn

**STOP. Ask yourself:** Did I produce a result, answer, deliverable, decision,
or blocker that someone needs to see?

If yes — **you MUST call `buzz messages send` before ending this turn.**
...
Checklist:
1. Was the user waiting on an answer? → buzz messages send
2. Did you finish delegated work? → buzz messages send (with callback @mention)
3. Did you find something worth reporting? → buzz messages send
4. Are you blocked and need input? → buzz messages send
5. Nothing to communicate? → Silence is correct. End without sending.
```

The design intent is "first and last" — the agent sees the CRITICAL banner immediately on session start, and is intercepted by the checklist before ending a turn.

### L3: Harness-level safety net (proposed design, not yet implemented)

The ACP harness already knows the target channel UUID (it's in the session context). After `session/prompt` returns with `EndTurn`, the harness should:

1. Query the relay: did the agent publish any events to the target channel since the turn started?
2. If no events were published AND the agent produced stdout output → log a warning + optionally auto-publish a fallback message
3. This catches the "agent produced output but forgot to publish" case regardless of agent type

Pseudo-code location: around `crates/buzz-acp/src/acp.rs` `session_prompt_with_idle_timeout` return path, or in the pool dispatch layer in `pool.rs`.

### Impact assessment

- **Without this fix**: Silent failures erode trust in agent reliability across the platform
- **L2 alone**: Reduces but doesn't eliminate the failure mode (prompt-based, not enforced)
- **L2 + L3**: Prompt reinforcement + infrastructure guarantee = defense in depth

Contributor guide

Open the contributing guide

Research direction

Read crates/buzz-acp/src/acp.rs, especially session_prompt_with_idle_timeout, and then inspect the pool dispatch layer in pool.rs. Trace how the harness knows the target channel and how stdout and relay events are handled after EndTurn. Done means detecting a turn with stdout but no published event, logging a warning, and determining whether a fallback publish is appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
devtools
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.