buzz-acp reports a turn as `ok` when the runtime refused to run and produced no output
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
### Summary
When a CLI runtime refuses to start a turn, it generally ends its ACP session cleanly without emitting any `agent_message_chunk`. `buzz-acp` maps every clean session end to `outcome="ok"`, posts nothing, and logs nothing above `DEBUG`. The agent looks healthy and idle while every turn is silently discarded.
There is currently no signal — log line, metric, or outcome label — that distinguishes *"the agent ran and had nothing to say"* from *"the agent never ran"*.
### Reproduction
Observed with `codex-acp` 1.1.7 / `codex-cli` 0.146.0 on a headless relay seat, but the failure is not codex-specific — any runtime that exits a turn without output behaves identically.
1. Run a `buzz-acp` seat whose working directory is not a trusted codex project (e.g. `WorkingDirectory=/home/codex`, the systemd default for a service user).
2. Mention the agent in a subscribed channel.
Result:
```
DEBUG buzz_acp: agent_claimed agent=0 channel=…
DEBUG buzz_acp: agent_returned agent=0 outcome="ok"
```
Elapsed: **~3.8 s**. No message posted. No warning at any level.
The underlying cause is visible only outside the harness:
```
$ codex exec "reply with the single word PONG"
Not inside a trusted directory and --skip-git-repo-check was not specified.
```
With the identical configuration and `WorkingDirectory` set to a trusted repository, the same prompt takes **~70 s** and replies normally. Both cases logged `outcome="ok"`.
### Why this is worth fixing
The same clean-but-empty ending is produced by several unrelated operational faults:
- expired or revoked CLI credentials
- an untrusted or non-repository working directory
- a missing or misconfigured provider (`OPENAI_COMPAT_*` and friends)
- a provider returning an error the adapter swallows
All of them present to an operator as *"the agent reacts with 👀 and then says nothing"*, with a success line in the log. On a multi-seat deployment the healthy seats keep working, so the failure reads as one agent being quiet rather than one agent being broken. Diagnosis currently requires reproducing the runtime invocation by hand outside the harness.
### Proposed fix
Track whether a turn emitted any non-whitespace assistant text, and report those turns separately:
- `AcpClient` gains a per-turn `turn_emitted_text` flag, reset at each `session/prompt` and set from `agent_message_chunk` in `handle_session_update`. Per-turn rather than per-session, because a runtime that refuses one prompt may serve the next — staleness would mask a recurring refusal.
- Whitespace-only chunks do not count, so a runtime that emits a stray newline before refusing still reports empty.
- `PromptOutcome::Ok(_)` with no emitted text is labelled `empty` instead of `ok`.
- Empty turns log at `WARN` rather than `DEBUG`. A default deployment does not run at `DEBUG`, which is precisely when this needs to be visible. The message names the three things worth checking: credentials, working directory, provider config.
No protocol change. No behaviour change for turns that produce output. The label flows through the existing `outcome` field, so anything already consuming turn metrics gets the distinction for free.
Diff is 86 insertions / 5 deletions across `crates/buzz-acp/src/acp.rs` and `crates/buzz-acp/src/lib.rs`, with a unit test covering the whitespace-only and non-empty cases. Full `buzz-acp` suite passes (650 tests).
Happy to open the PR if the approach looks right.
### Possible follow-ups (not included)
- Escalate after N consecutive `empty` turns for one agent — a single empty turn can be legitimate, a run of them is not.
- A pre-flight probe at pool start, so a seat that can never produce output fails loudly at startup instead of on first mention. This would also cover adapters that fail before any session is created.
- Arguably `codex-acp` should surface the CLI's refusal as an ACP error rather than an empty successful turn. The harness-side fix is still worth having as defence in depth, since it covers every runtime rather than one.
Contributor guide
Assessment
This issue has not been assessed yet.