unic / unic/unic-agents-plugins
The two-axis review does not ask the question that eleven of PR #461's fifteen defects answered to
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 19
Description
What this is
PR #461 took seven Copilot rounds, two local /code-review passes, one coherence sweep and one Consumer run to reach green. Roughly fifteen defects surfaced. Every one was real; none reached develop. The process worked, and it worked at a cost worth reducing.
This ticket is about the cost. The defects fall into classes, the classes are not evenly served by the checks this repo runs, and one class dominates.
What the fifteen were, and who caught them
| Class | Count | Found by |
|---|---|---|
| Local self-inconsistency — a schema requiring nine keys beside a prompt saying "pass through what you read"; a comment header describing the node above it; a table row contradicting its own second sentence; a count written beside a list that changed length; one instruction stating the same rule three times, two of them disagreeing; a sentence left mangled because an edit landed beside the text it should have replaced | 11 | Copilot found most. The local /code-review found none of them on its first pass, and found three on a later pass on the same tree, after a third question was added to its prompt |
Cross-file drift — five surfaces still describing /qa without a node this PR added; an ADR guaranteeing behaviour written before the floor existed |
4 | local /code-review, and a hand sweep the maintainer asked for |
Behavioural — a prompt wording that would have made a branch unreachable; a $node.output.<field> reference to a field the node is instructed to omit; a verdict printed to a stream that also carried prose, so it never parsed |
3 | split between both |
| Spec fidelity | — | caught during grilling, before implementation |
Eleven of fifteen are one class, and it is the class the two-axis review does not ask about. Its two axes ask "does this follow documented standards" and "does this match the spec". Neither asks "is this file consistent with itself".
Three of those eleven were introduced by the fix for the round before, which is the second finding here: rounds 2 and 3 and the mangled sentence are one failure mode, not three defects.
Proposals
Ranked by the evidence above. Not pre-decided — the fourth carries a real tension and the grilling should settle it.
1. A third axis on /code-review: self-consistency
.claude/skills/code-review/SKILL.md defines two axes. Add a third that reads each changed file against itself: an output schema against the prompt that fills it, a comment header against the node beneath it, a table row against the behaviour it describes, a stated count against the list beside it, a rule against every other statement of the same rule in the same prompt.
Evidence it works: this axis was added by hand to a re-run of the same review, on the same commit the two-axis version had already passed, and it immediately returned three defects — including a sentence with a duplicated clause and a dangling verb, sitting in a prompt an agent executes at run time.
2. Run the review after the last fix, not only before the first push
Four of Copilot's findings were present in the exact commit the local review read. A one-shot review of a 700-line prose diff under-samples; the review is cheap and the second pass is where the churn shows.
3. A stated rule for how a fix is applied: replace, never append
Rounds 2 and 3 and the mangled step are the same failure — a correction landed beside the text it should have replaced, leaving two statements that disagree, or one sentence carrying half of each. Root AGENTS.md already tells a reviewer to read their diff with the unchanged lines around each edit in view. What is missing is the paragraph-level instruction to the editor: after fixing, re-read the edited paragraph whole, against itself, before moving on. This is the only proposal here that prevents rather than detects.
4. A CI check for dangling $node.output.<field> references — and the tension it carries
A reference to a field a node is instructed to omit is the class that kills an Archon run at template resolution with no useful error. It is mechanically checkable: parse each Box YAML, collect every $<node>.output.<field> token, and confirm the named node's output_format declares that field.
The tension, stated rather than hidden. Root AGENTS.md names "adding a module plus a test so that prose becomes testable" as a defect, and #381 deleted exactly that from unic-archon-dlc. Two things distinguish this from what #381 removed: it would live in the repo's CI and never install into a Consumer, and it validates a reference graph, not prose. It is still code watching prose, and the grilling should decide whether that line holds. If the answer is no, say so and close this proposal — a stated ceiling beats a check nobody trusts.
5. Give Copilot the repo's own conventions
Copilot's review offers .github/skills/code-review/SKILL.md as a way to feed it repo context. Today it reviews with none, which is why several of its rounds re-raised decisions this repo had already settled and recorded. Cheap; may also shorten the rounds.
Not proposed, deliberately
- Fewer review rounds. Every round here found something real. The cost is worth paying; this ticket is about paying less for the same result.
- Dropping the Consumer run. It found no new defect on #461, but it is the only check that runs a node, and it proved three acceptance criteria that nothing in this repository can observe.
What this did not decide
- Whether proposal 4 crosses the no-code line, and if it does, whether anything replaces it.
- Whether the third axis is a new agent or a widening of the Standards axis. A third agent costs a third fan-out; widening risks the same under-sampling that missed eleven defects.
- Whether proposal 3 belongs in root
AGENTS.md, in thecode-reviewskill, or in both.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .claude/skills/code-review/SKILL.md and root AGENTS.md first, then inspect the Box YAML references described in the issue. Compare the five proposals and the unresolved trade-offs, including the boundary identified by #381. Done means one approach is agreed, its affected review or CI guidance is updated, and the stated acceptance questions are settled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- markdown, yaml
- Domain
- developer-experience, testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100