awslabs / awslabs/aidlc-workflows

[Feature]: default the balanced tier to `effort: medium` — reviewer-shaped work pays session-effort latency without verdict gains

Open
#607 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
4.6k
Forks
827
Avg merge
1d 21h
Merged PRs (30d)
165

Description

### Description

The 2.3.1 tier system (PR #546, closing #533) projects the `balanced` tier — `aidlc-architecture-reviewer-agent` and `aidlc-product-lead-agent` — as `model: sonnet` with **no effort pin**, so the session's reasoning effort carries into every delegated review (`core/tools/aidlc-tiers.ts`, `TIER_PROJECTIONS.balanced`: claude `{model: "sonnet", effort: null}`; same omission on codex/opencode; Kiro rides `chat.modelDefaults` at `high`).

Proposal: step the `balanced` tier down to `effort: medium` in `TIER_PROJECTIONS` — the same deliberate downgrade `templated` already carries — so reviewer-shaped work stops paying session-effort latency:

- Claude Code: `effort: medium` line in the two projected agent .md files
- Codex: `model_reasoning_effort = "medium"` in the two role TOMLs
- opencode: `variant: medium`
- Kiro: `KIRO_TIER_EFFORT.balanced` `high` → `medium` (collapse-rule consequence: balanced and templated share `claude-sonnet-4.5`, so the single `chat.modelDefaults` entry becomes `medium`)

This deliberately revisits the "balanced keeps session effort" decision from the #533 thread, with field data that was not available then.

### Use Case

Field data from a 12-unit intent (AIDLC 2.2.17 on Claude Code, session model Fable 5 at `xhigh`):

- The restarted U01 nfr-design review spent **~36 of 39 minutes in time-to-first-token** at ~140k context under xhigh. Effort multiplies per-call latency with context size, and §12a reviews carry the largest contexts in the workflow (unit artifacts + Q&A + shared inception contracts).
- **11 of 12** nfr-requirements reviews returned READY. The extra reasoning depth was buying latency, not verdicts — consistent with the tier's own definition ("novel input judged against explicit criteria"): since 2.3.5 the review contract is checklist- and evidence-grounded (validation tools, acceptance criteria, passed contracts), which is exactly the shape of work that does not benefit from maximum-effort deliberation.
- The project applied `effort: medium` to its installed reviewer agents (the field predecessor of this proposal) and verified via a probe subagent that a pinned `effort:` in Claude Code agent frontmatter overrides the session value (PreToolUse payload carried `effort.level=medium`). Subsequent reviews completed in a fraction of the wall-clock with no observed verdict-quality regression across the remaining design stages.

Why change the shipped default rather than add another knob: end users copy `dist//` and never run the packager, so a pack-time knob (like the existing `tier_cap:`) only reaches source builds — the shipped default is what users actually get. The escape hatch for a user who wants session effort back on reviews already exists and is documented: edit the projected value in the installed copy (`docs/reference/05-agent-system.md`, "Change tier" — here, delete the `effort:` line in `.claude/agents/aidlc-architecture-reviewer-agent.md` / `aidlc-product-lead-agent.md`).

Scope guard: `judgment` agents are untouched — the "never silently downgrade the session" contract still holds where output cascades downstream. This narrows the session-effort inherit to the tier whose work product is a bounded verdict.

### Version

v2 (alpha)

### Area

Other

### Additional Context

Implementation sketch (verified against v2 @ 2.5.0, commit 3f16023):

- `core/tools/aidlc-tiers.ts` — `TIER_PROJECTIONS.balanced` (claude/codex effort, opencode variant) + `KIRO_TIER_EFFORT.balanced`; update the header comment block's tier rationale ("balanced ... session effort" → medium) and the Kiro collapse comment (the shipped entry becomes sonnet-4.5 → medium).
- Test repins: `tests/unit/t216-agent-tier-projection.test.ts` (balanced `effort: null` expectations + the "effort ABSENT everywhere else" case), `tests/unit/t220-tier-projection-module.test.ts` (`EXPECTED.balanced` all four harness flavors, `kiroModelDefaults()`, `KIRO_TIER_EFFORT.balanced === "high"`, the codex shipped-bytes omit-effort case), `tests/smoke/t148-kiro-file-structure.test.ts` (cli.json `claude-sonnet-4.5 → high` pin).
- Docs: `docs/reference/05-agent-system.md` (tier table + projection table), `docs/reference/14-claude-features.md`, `docs/guide/06-agents.md`.
- Regenerate dists (`bun scripts/package.ts` + `--check`); CHANGELOG + version bump per policy.

Pack-time only — no runtime-engine change; `tier_cap:` semantics unchanged (it caps the model axis; this sets the balanced tier's effort dial).

Related: #533 / PR #546 (the tier system this tunes); #534 / PR #538 (reviewer read-scope — the same field runs motivated both; the read-scope bound limits WHERE the reviewer reads, this limits how expensively it thinks). A separate issue proposes a turn backstop for the reviewer — independent change, same cost theme.

Contributor guide

Open the contributing guide

Research direction

Start with core/tools/aidlc-tiers.ts and compare the balanced projections with the existing templated values. Run the named projection and Kiro tests in tests/unit/t216-agent-tier-projection.test.ts, tests/unit/t220-tier-projection-module.test.ts, and tests/smoke/t148-kiro-file-structure.test.ts, then review the listed reference docs. Done means the expectations, documentation, regenerated distributions, changelog, and version are consistent after bun scripts/package.ts and its --check pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
ai-infra-agents, build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.