get2knowio / get2knowio/maverick

Contrarian briefing agent repeatedly fails structured output on haiku

Open
#166 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4
Forks
0
Avg merge
17h 37m
Merged PRs (30d)
7

Description

Found during the #135 subtask 5 live walkthrough of sample-maverick-project.

The contrarian briefing agent failed **two consecutive runs** on the same
flight plan with:

```
airframe.errors.AgentRuntimeError: claude_code: is_error
subtype=error_max_structured_output_retries
['Failed to provide valid structured output after 5 attempts']
```

Config: `agents.briefing` bound to `claude` / `claude-haiku-4-5-20251001`.
Navigator, Structuralist and Recon — the same role, same binding, same
run — all succeeded every time.

## Why this probably isn't a plain flake

It succeeded once (254.7s) and then failed twice in a row on the same
input. Five internal retries each time. That pattern points at the
contrarian's own payload schema or prompt being hard for haiku to
satisfy, rather than provider noise.

The contrarian differs from its three siblings in one way that matters:
its prompt embeds the *other three briefs* (`build_contrarian_prompt`
takes navigator/structuralist/recon), so it runs against a much larger
context and is asked for a different payload shape
(`SubmitContrarianBriefPayload.challenges`).

## Suggested investigation

1. Reproduce with `MAVERICK_LOG_LEVEL=DEBUG` and capture what haiku
actually returned on each of the 5 attempts.
2. Compare `SubmitContrarianBriefPayload`'s schema complexity against
the three that succeed.
3. If it is a model-capability limit, the honest fix is a per-role tier
for the contrarian rather than pretending haiku can serve it — the
plumbing for that now exists (`actors.refuel.decomposer.tiers` and
friends, wired in #165).

## Not blocking

`c4fec21` makes briefing failures non-fatal, so a contrarian failure now
costs its brief rather than the run. This issue is about the underlying
failure, not the blast radius.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Research direction

Reproduce the contrarian briefing with MAVERICK_LOG_LEVEL=DEBUG, then inspect build_contrarian_prompt and SubmitContrarianBriefPayload.challenges alongside the schemas used by Navigator, Structuralist, and Recon. Compare the five returned attempts and determine whether the underlying failure is resolved or whether a per-role tier using actors.refuel.decomposer.tiers is needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.