BOHICA-LABS / BOHICA-LABS/vsdd-factory

bug(adversary): agent cannot persist its own review report; round-trips through orchestrator

Open
#211 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

The `adversary` agent has a read-only tool profile (Read, Grep, Glob). When it produces its review report, it can only return the report inline in its final assistant message. The orchestrator must then dispatch `state-manager` separately to persist the report to disk at `.factory/cycles//adversarial/p-pass-.md`.

This adds ~30k tokens of round-tripping per adversarial pass (the report content travels orchestrator → state-manager → disk) and creates a risk: if the orchestrator forgets to dispatch state-manager, or state-manager fails mid-execution, the report is lost.

## Reproduction

Every adversarial pass run during the ftc-blue Phase 1d cycle (passes 5–9, 2026-06-23):

1. Orchestrator dispatches adversary with fresh context.
2. Adversary produces a structured frontmatter + Markdown report (150–250 lines).
3. Adversary's final message contains the entire report verbatim, with instructions to the orchestrator to persist it.
4. Orchestrator dispatches state-manager with the full report verbatim in the prompt.
5. State-manager writes the file and commits it.

Each pass: ~30k tokens to round-trip the report through the orchestrator. Five passes = ~150k tokens spent on persistence overhead alone.

In one observed case (pass-8 finalization on ftc-blue), state-manager hit "API Error: Connection closed mid-response" mid-execution. The orchestrator had to detect the failure, compose a retry, and re-dispatch — adding more friction.

## Proposed change

Two options:

**Option A — Expand adversary tool profile.**

Add a narrowly-scoped `Write` capability to the adversary, restricted to a single predetermined path: `.factory/cycles//adversarial/p-pass-.md`. The path can be computed from the agent's input context (cycle, phase, pass) without requiring shell access. No general write capability — just one file.

**Option B — Post-completion hook.**

Add a hook that runs after the adversary agent completes. The hook reads the agent's final message, extracts the report content (delimited by the YAML frontmatter at the top), and writes to the canonical path. Orchestrator never needs to handle the report content.

Option B is preferred — keeps the agent's tool surface unchanged, moves persistence out of orchestrator scope, and consolidates the path computation in one place.

## Applies to

- `agents/adversary.md` — tool profile (Option A) or hook integration (Option B)
- `hooks-registry.toml` — register the persistence hook (Option B)
- `agents/orchestrator/` — remove the "now dispatch state-manager to persist" workflow step

## Acceptance criteria

- [ ] Adversary report lands at canonical path without orchestrator intervention
- [ ] Round-trip cost per pass reduced to ~0 tokens
- [ ] State-manager dispatch for "persist adversary report" no longer needed
- [ ] Failure mode: if persistence fails, the failure surfaces immediately (not silently)

## Found during

ftc-blue Phase 1d adversarial cycle, passes 5–9 (2026-06-23, vsdd-factory@1.0.0-rc.21).

## Notes

The same pattern likely applies to other read-only review agents (`consistency-validator`, `holdout-evaluator`, `code-reviewer`, `pr-reviewer`). A general "agent-output-persistence" hook framework would benefit all of them.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.