unic / unic/unic-agents-plugins

The two-axis review does not ask the question that eleven of PR #461's fifteen defects answered to

Open
#462 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs-specs repo tech-debt
Dominant language
JavaScript
Stars
1
Forks
0
Avg merge
16h 43m
Merged PRs (30d)
19

Description

What this is

PR #461 took seven Copilot rounds, two local /code-review passes, one coherence sweep and one Consumer run to reach green. Roughly fifteen defects surfaced. Every one was real; none reached develop. The process worked, and it worked at a cost worth reducing.

This ticket is about the cost. The defects fall into classes, the classes are not evenly served by the checks this repo runs, and one class dominates.

What the fifteen were, and who caught them

Class Count Found by
Local self-inconsistency — a schema requiring nine keys beside a prompt saying "pass through what you read"; a comment header describing the node above it; a table row contradicting its own second sentence; a count written beside a list that changed length; one instruction stating the same rule three times, two of them disagreeing; a sentence left mangled because an edit landed beside the text it should have replaced 11 Copilot found most. The local /code-review found none of them on its first pass, and found three on a later pass on the same tree, after a third question was added to its prompt
Cross-file drift — five surfaces still describing /qa without a node this PR added; an ADR guaranteeing behaviour written before the floor existed 4 local /code-review, and a hand sweep the maintainer asked for
Behavioural — a prompt wording that would have made a branch unreachable; a $node.output.<field> reference to a field the node is instructed to omit; a verdict printed to a stream that also carried prose, so it never parsed 3 split between both
Spec fidelity caught during grilling, before implementation

Eleven of fifteen are one class, and it is the class the two-axis review does not ask about. Its two axes ask "does this follow documented standards" and "does this match the spec". Neither asks "is this file consistent with itself".

Three of those eleven were introduced by the fix for the round before, which is the second finding here: rounds 2 and 3 and the mangled sentence are one failure mode, not three defects.

Proposals

Ranked by the evidence above. Not pre-decided — the fourth carries a real tension and the grilling should settle it.

1. A third axis on /code-review: self-consistency

.claude/skills/code-review/SKILL.md defines two axes. Add a third that reads each changed file against itself: an output schema against the prompt that fills it, a comment header against the node beneath it, a table row against the behaviour it describes, a stated count against the list beside it, a rule against every other statement of the same rule in the same prompt.

Evidence it works: this axis was added by hand to a re-run of the same review, on the same commit the two-axis version had already passed, and it immediately returned three defects — including a sentence with a duplicated clause and a dangling verb, sitting in a prompt an agent executes at run time.

2. Run the review after the last fix, not only before the first push

Four of Copilot's findings were present in the exact commit the local review read. A one-shot review of a 700-line prose diff under-samples; the review is cheap and the second pass is where the churn shows.

3. A stated rule for how a fix is applied: replace, never append

Rounds 2 and 3 and the mangled step are the same failure — a correction landed beside the text it should have replaced, leaving two statements that disagree, or one sentence carrying half of each. Root AGENTS.md already tells a reviewer to read their diff with the unchanged lines around each edit in view. What is missing is the paragraph-level instruction to the editor: after fixing, re-read the edited paragraph whole, against itself, before moving on. This is the only proposal here that prevents rather than detects.

4. A CI check for dangling $node.output.<field> references — and the tension it carries

A reference to a field a node is instructed to omit is the class that kills an Archon run at template resolution with no useful error. It is mechanically checkable: parse each Box YAML, collect every $<node>.output.<field> token, and confirm the named node's output_format declares that field.

The tension, stated rather than hidden. Root AGENTS.md names "adding a module plus a test so that prose becomes testable" as a defect, and #381 deleted exactly that from unic-archon-dlc. Two things distinguish this from what #381 removed: it would live in the repo's CI and never install into a Consumer, and it validates a reference graph, not prose. It is still code watching prose, and the grilling should decide whether that line holds. If the answer is no, say so and close this proposal — a stated ceiling beats a check nobody trusts.

5. Give Copilot the repo's own conventions

Copilot's review offers .github/skills/code-review/SKILL.md as a way to feed it repo context. Today it reviews with none, which is why several of its rounds re-raised decisions this repo had already settled and recorded. Cheap; may also shorten the rounds.

Not proposed, deliberately

  • Fewer review rounds. Every round here found something real. The cost is worth paying; this ticket is about paying less for the same result.
  • Dropping the Consumer run. It found no new defect on #461, but it is the only check that runs a node, and it proved three acceptance criteria that nothing in this repository can observe.

What this did not decide

  • Whether proposal 4 crosses the no-code line, and if it does, whether anything replaces it.
  • Whether the third axis is a new agent or a widening of the Standards axis. A third agent costs a third fan-out; widening risks the same under-sampling that missed eleven defects.
  • Whether proposal 3 belongs in root AGENTS.md, in the code-review skill, or in both.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read .claude/skills/code-review/SKILL.md and root AGENTS.md first, then inspect the Box YAML references described in the issue. Compare the five proposals and the unresolved trade-offs, including the boundary identified by #381. Done means one approach is agreed, its affected review or CI guidance is updated, and the stated acceptance questions are settled.

Written by the indexing model from the issue text.

Assessment

Tech stack
markdown, yaml
Domain
developer-experience, testing-qa, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.