feat(herdr): Select adequate agents and reasoning effort by task evidence and total cost

Problem

Herdr needs to select the least costly agent demonstrably adequate for each assignment, with model and reasoning effort chosen separately. Existing tier support does not achieve this when the roster uses flat defaults, and several policy rules encourage broad use of expensive models and effort levels.

A September 16 ACR session audit found Astra High, not Ultra, in the inspected Codex sessions. It nevertheless showed Astra handling routine coordination and release work, with repeated polling, report reads, and large context contributing overhead. In one approximately 41-minute lead segment, counters increased by about 270k uncached input tokens, 16.1M cached input tokens, and 46k output tokens. Cached input includes repeated context; these are usage counters, not dollar charges or attributable shared-quota costs.

Relevant current behavior:

• rules/agent-team-operation.md requires the lead to use the strongest generally available model and pins judgment rounds to top models.
• skills/herdr-teamlead/teamlead/tiers.py escalates hostile verification to extra-high effort and also uses context size as an escalation trigger.
• The audited roster mostly used non-tiered model defaults, so tier selection was not managing those assignments.
• Planner ranking emphasizes projected subscription headroom. Available quota does not establish that a stronger model is necessary or cheaper through task completion.
• Qualification currently requires a five-case screen, twenty-case promotion battery, and weekly five-case canary per model/effort/role. Its operational cost and suitability for different responsibilities need evaluation.

Proposed policy

Select agents by task requirements, demonstrated competence, and independence. Among adequate candidates, prefer the lowest expected total resource use, including retries and coordination. Choose model and reasoning effort separately. Higher effort requires a task-specific reason; role name, available quota, and large context alone are insufficient.

1. Separate mechanics from judgment

• Use owner scripts for deterministic polling, ledger updates, and CI result collection.
• Route bounded coordination and release mechanics to qualified economical workers at appropriate effort.
• Keep acceptance decisions, disputed findings, and other judgment explicit; do not relabel them as mechanics to obtain a cheaper assignment.
• Escalate difficult decisions to a qualified specialist instead of running every administrative operation through the strongest lead model.

2. Make escalation evidence-based

• Record the unresolved question, observed failure, or interacting risks that justify stronger models or higher effort.
• Replace automatic extra-high selection based solely on the tester role with task-specific qualification and risk requirements.
• Before escalating because of context size, improve task packaging where possible without dropping required review scope or evidence.
• Re-evaluate the required tier at the next suitable assignment boundary. Preserve cumulative correction counts and retained-session contracts.

3. Make qualification usable without weakening gates

Evaluate reusable, task-specific qualification evidence proportional to the responsibility. Mechanical release execution, implementation, and independent defect detection need different demonstrations of competence. Define evidence validity and invalidation explicitly; never manufacture qualification receipts or equate a green CI run with reviewer competence.

This changes policy established in #324, not merely configuration. That issue records failed lower-tier controller/reviewer experiments and warns against repeating them without a structurally different design. Any proposal to lower judgment floors must address that evidence and demonstrate preserved defect detection and correct severity before live adoption. Keep the pinned judge contract in the initial rollout.

4. Explain and measure each assignment

Plans should record required capabilities, independence exclusions, selected model/effort, qualification evidence, the reason cheaper candidates were inadequate (or cost remains unknown), and the escalation condition.

Use headroom as a capacity constraint. Compare total resource use through acceptance, including failed attempts, rather than token price alone. Report uncached input, cached input, output, elapsed time, correction rounds, and coordination overhead separately. Preserve shared billing-window accounting and unknown attribution.

Expose flat/default routing explicitly and provide a validated migration to tiered configuration; do not present legacy assignments as cost-optimized selections.

Rollout and acceptance

• Update the rule, skill guidance, tier selection/planner behavior, and configuration example together.
• Start with deterministic automation, bounded coordination, and release mechanics; keep current independent reviews, tests, external review, and release checks required.
• Demonstrate valid task-specific qualification before lowering any reviewer/tester judgment floor.
• Verify actual launch model and effort; preserve worker identity, task history, independence, and correction allowances.
• Test routing boundaries: ordinary mechanics avoid unnecessary escalation, genuine uncertainty escapes mechanical routing, insufficient qualification prevents dispatch, and legacy routing is visibly identified.
• Test that task-specific escalation and subsequent assignment-level downshifting preserve all quality gates.
• Compare representative completed tasks against the current baseline, reporting quality, retries, context/coordination overhead, elapsed time, and attributable cost where available. Do not claim savings from shared quota changes alone.

Related work

• #324 implemented per-round tiers; this proposal builds on that machinery and explicitly revisits selected policy constraints.
• #321 contains the original research and qualification rationale.
• #434 is ongoing review-seat work touching the same implementation surfaces; coordinate changes without overwriting it.

AI disclosure: Drafted and filed with Codex at the repository owner's request, based on local policy/planner inspection and the session usage audit. No policy implementation or worker-model changes are included in this issue.

Start by reading rules/agent-team-operation.md and skills/herdr-teamlead/teamlead/tiers.py, then trace the planner, tier-selection, and configuration-example surfaces named in the issue. Review #324 and #321 for existing qualification constraints and coordinate with #434. Done means the policy, guidance, routing, configuration, and boundary checks are updated together while preserving the listed quality gates and reporting requirements.