unic / unic/unic-agents-plugins
bug(unic-pr-review): intent-check merger discards all verdicts when an item has a `note` — Intent Check silently renders as all-unaddressed
@orioltf is already working on this.
Since Aug 11, 2026.
- Dominant language
- JavaScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 19
Description
Summary
intent-check-merger.mjs short-circuits on any skeleton item that carries a note, returning it with verdicts untouched. Because the Intent Checker emits note on ordinary items (to record AC provenance), a fully-assessed Intent Check is silently discarded and every AC renders as unaddressed.
This reproduces the exact symptom of the closed #160 ("Intent Check verdicts are static (always unaddressed)") through a different mechanism.
Root cause
apps/claude-code/unic-pr-review/scripts/lib/intent-check-merger.mjs:102
const items = skeleton.map((skeletonItem) => {
if (skeletonItem.note !== undefined) {
return skeletonItem // <-- verdicts never overlaid
}
...
The module header documents this as the ADR-0004 hard-stop passthrough:
*items (ADR-0004 hard-stop signal) pass through verbatim, verdicts untouched.
…and agents/intent-checker.md:59 documents note only for the not-found soft-error case:
{ "id": "<item.id>", "title": "Work Item <item.id>", "verdicts": {}, "note": "Work Item not found." }
But note is a free-text field on a schema handed to an LLM agent, and the natural reading is "annotate this item". In practice the Intent Checker uses it for legitimate provenance on healthy items. From a real run against ADO PR 5647 (4 linked work items, all reachable, all assessed):
"note": "No AcceptanceCriteria field on the work item; ACs enumerated from Feature FPTD-01-b on Confluence page 603390377. AC 2, AC 3 and AC 4 are struck through in the work item's ToDo list and are descoped."
All four items carried a note of this kind. So note is overloaded: it means both "this item is a hard-stop, do not touch" and "here is context about this item".
Reproduction
SKELETON_JSON='[{"id":"1","title":"T","verdicts":{"AC 1":"unaddressed"},"note":"ACs enumerated from Confluence page 12345."}]' \
ASSESSED_JSON='[{"id":"1","verdicts":{"AC 1":"addressed"}}]' \
node apps/claude-code/unic-pr-review/scripts/lib/intent-check-merger.mjs
Actual
{"items":[{"id":"1","title":"T","verdicts":{"AC 1":"unaddressed"},"note":"..."}],
"diagnostics":{"assessedReceived":1,"applied":0,"droppedElements":0,"rejectedVerdicts":0,"unmatchedItems":0}}
Expected — AC 1 overlaid as addressed, applied: 1.
Dropping the note key from the same input yields applied: 1, confirming note is the sole trigger.
Impact
Silent and total, and it degrades in the most misleading possible direction — a reviewer sees a plausible, fully-populated Intent Check block asserting that the PR addresses none of its acceptance criteria.
In the PR 5647 run this suppressed all 41 verdicts across 4 work items. The correct output was 25 addressed / 11 partially addressed / 5 unaddressed. The first render claimed 0 / 0 / 41. It was caught only because applied: 0 looked wrong against an Assessor that had visibly returned verdicts; nothing in the rendered summary flagged it.
Note the diagnostics cannot distinguish this failure from a legitimate one: droppedElements, rejectedVerdicts and unmatchedItems are all 0, exactly as in a healthy run where the Assessor genuinely returned nothing. So the maintainer-facing stderr drift classes in review-pr.md Step 8 item 4 do not fire either.
The module's own header names this risk precisely:
a wholesale silent fallback to
unaddressedis indistinguishable, to the Reviewer, from a genuine "the diff doesn't address this"
There is a partial safety net — review-pr.md Step 8 item 3 requires the orchestrator to set unassessedIntentCheck: true when applied === 0, which raises a reviewer-facing Notice. It depends on the orchestrator checking applied, and it reports the wrong cause even when it fires (it says the Assessor produced no valid verdicts; the Assessor was fine).
Suggested fix
The overload is the bug; note should not be a control signal. Options, roughly in order of preference:
- Use an explicit field for the hard-stop. Have the Intent Checker set something like
"hardStop": true(or"unreachable": true) on items that must pass through untouched, and key the short-circuit on that.notethen goes back to being free-text documentation. Requires a matching change inagents/intent-checker.md. - Key the short-circuit on the actual hard-stop shape —
Object.keys(skeletonItem.verdicts).length === 0. An item with no ACs to assess has nothing to overlay, which is the real invariant behind the not-found case, and it does not care whatnotesays. - At minimum, make it loud. Count short-circuited items in
diagnostics(e.g.passthroughItems) soapplied: 0withpassthroughItems: 4is distinguishable from a genuinely empty Assessor response, and extend the Step 8 stderr drift classes to name it.
(2) is the smallest change and fixes the reported case; (1) is the one that stops this recurring.
A regression test with a note on a healthy, fully-assessed item would have caught this — worth adding alongside whichever fix lands.
Environment
| Plugin | unic-pr-review 2.1.12 |
| Repo commit | d7dd452 |
| Node | v22 |
| Observed in | /unic-pr-review:review-pr against ADO PR (re-review, 4 linked work items, 41 ACs) |
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.