aaif-goose / aaif-goose/goose

Hooks: add non-blocking context injection + tool-execution trace to the Stop payload (enables the #9708 completion gate without deadlock)

未关闭
#10,358 0 条评论 0 个 reaction 已指派 1 人 已被 @alexhancock 认领 在 GitHub 查看
主要语言
Rust
星标
54.2k
派生
6.2k
平均合并
3 天 2 小时
30 天内合并 PR
262

描述

## Context

This follows directly from #9708 (ground-truth completion gate — verify an agent's success claims against actual tool execution before a turn can end). I built a working prototype of that gate as an external verifier that runs on the `Stop` hook: at turn-end it audits the agent's final message against what actually executed, and if the agent claimed success it hasn't earned (e.g. "all tests pass" with no test run in the trace), it pushes back.

I ran it live against a cheap agentic model (DeepSeek V4 Flash via goose 1.41.0, openrouter) on a long, context-heavy coding task, using the current blocking-hook mechanism (`exit 2` + reason on stderr).

## The finding: block-only makes the gate deadlock

Wired as a **blocking** Stop hook, the completion gate recreates the exact failure it's meant to catch. The agent claims done → the hook blocks ("that claim isn't backed by a verification receipt") → the agent re-claims → blocked again → repeat. In one run this ran **70 consecutive blocks with zero forward progress** before I killed it. `GOOSE_STOP_HOOK_BLOCK_CAP` bounds the loop, but the cap only chooses between two bad ends: cap low and the gate gives up and lets the confabulation through; cap high and you get an N-deep deadlock. Either way the agent is stuck, because it often **cannot** produce the receipt the gate demands within that turn (the check is expensive, or it genuinely doesn't know how) — and a wall is the wrong response to "you haven't verified this yet."

The right response to an unverified claim is almost never "you may not finish" — it's "here's what's unverified; address it or say so." That is an **advisory** signal, and goose hooks currently can't emit one.

## Root cause: hooks can BLOCK or OBSERVE, but not ADVISE

Today (`crates/goose/src/hooks/mod.rs`) a hook's influence on the model is binary:
- **Block** — `{"decision":"block","reason":"..."}` on stdout, or `exit 2` with a reason on stderr (`emit_blocking`). Hard-stops the turn.
- **Observe** — any other event fires the hook for its side effects; its output is discarded.

There is no channel to feed a hook's finding **into the model's next turn without blocking it**. The output parser reads only `decision`; there is no `additionalContext`. So the completion gate can wall the agent (deadlock) or log to a file the agent never sees (useless) — nothing in between.

## Proposal (three additive, backward-compatible primitives)

1. **Non-blocking context injection.** Let a hook return, on stdout:
```json
{"hookSpecificOutput": {"hookEventName": "Stop", "additionalContext": "veritaserum: 'all tests pass' has no test run in this turn's trace — verify or retract."}}
```
goose injects `additionalContext` into the model's context (system-note style) and the turn proceeds. This is the missing "advise" tier — the gate warns, the agent sees it next turn, no deadlock. Wire it on `UserPromptSubmit` and `Stop` (and it composes with the existing `decision` field for hooks that still want to hard-block).

2. **Tool-execution trace in the `Stop` payload.** #9708 is literally "verify success claims against actual tool execution." A gate can't do that without the executions. `HookContext` already carries `last_assistant_message` + `working_dir`; add the turn's tool-call/result trace (tool name, args summary, exit/result) so the hook can check the claim against what ran, without reverse-engineering `sessions.db`.

3. **A `warn` decision.** Alongside `block`/allow, a `{"decision":"warn","reason":"..."}` that surfaces the reason to the model (via the same injection path) without denying turn-end. Most of what a verification gate should do is warn, not wall — the block tier stays for the rare hard-stop.

All three are additive: existing hooks (block/observe) are unaffected; only hooks that opt into the new fields see new behavior.

## Prior art: other harnesses already have the advise tier

This isn't novel — it's parity with what verification/policy tooling relies on elsewhere:

- **Claude Code** hooks return `hookSpecificOutput.additionalContext` on `UserPromptSubmit`/`PostToolUse`; the string is injected into the model's context (wrapped as a system reminder), non-blocking. Its `Stop` hook supports `{"decision":"block","reason":...}` (which goose matches) with a `stop_hook_active` flag to prevent the exact infinite-block loop we hit — goose's `GOOSE_STOP_HOOK_BLOCK_CAP` is the analogue.
- **OpenAI Codex** injects `additionalContext` on `SessionStart`, `PreToolUse`, `PostToolUse`, `UserPromptSubmit`, **and `Stop`** — and its `Stop`/`SubagentStop` hooks use `continue: false` to *request additional processing* (i.e. send the agent back to correct) rather than hard-halt the turn. That is exactly the advise-and-let-it-correct tier proposed here, already shipping.

The pattern across both major harnesses is the same: a Stop-class hook can **advise the model and let it correct**, not only deny it. goose today has only the deny half. This adds the advise half (and the tool-trace the gate needs to advise usefully) — bringing goose's hook surface to parity and making #9708 implementable as a gate rather than a wall.

## Why this unblocks #9708

The completion gate in #9708 becomes implementable *as designed* — a gate, not a wall:
- **Verify**: (2) gives it the tool trace to check claims against.
- **Correct without deadlock**: (1)/(3) let it feed "this claim is unverified" into the next turn; the agent re-runs or retracts on its own; no forced re-turn, no runaway.

I have the external verifier and the live deadlock repro; happy to contribute the hook-side changes (I already added `working_dir` to the Stop context on my fork). PRs to follow.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。