unic / unic/unic-agents-plugins

spike(unic-archon-dlc): the review Box is not deterministic across iterations on identical code

Open
#485 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

app:unic-archon-dlc needs-specs p2 spike wayfinder:grilling
Dominant language
JavaScript
Stars
1
Forks
0
Avg merge
16h 43m
Merged PRs (30d)
19

Description

Observed

Run 3, two unic-dlc-pr-review iterations on PR 5881 at the same commit 08e2523. Iteration 1: 14 findings (Standards 9, Spec 5), deduped to 13. Iteration 2: 13 findings (Standards 3, Spec 10). 0 of 13 prior hashes reappeared. By hand: 10 are the same defects reworded, re-anchored or axis-flipped, 3 are new (all Spec, all acceptance-criteria reads iteration 1 did not do), 6 vanished (all Standards). Praise: 13 bullets each; about 7 survive reworded, 3 file-and-line bullets vanished, one new bullet praises what iteration 1 filed as a defect.

The hash includes the normalised title (unic-dlc-pr-review.yaml:354), so rewording alone explains the 0/13; reconcile's semantic fallback (:448) matched 7. Run 2 reproduced six hashes, which now reads as less rewording, not determinism.

Why it matters

The register's run-2 comparables (dedup, reconcile, praise) assume a stable reviewer. Run 3 measured the variance once; run 4's seal needs a baseline before any fixed is read as signal (#{G4}).

Evidence

Not decided

  • How many identical-code iterations make a variance baseline, and what the Box should report when the axis split inverts.
  • Whether the Standards axis should be seeded with the prior iteration's findings as things to confirm or retract, rather than re-derived blind.

Filed 2026-09-17 by wayfinder-orchestrator-10 from run 3's measurements. needs-specs: grill before dispatch.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with unic-dlc-pr-review.yaml at lines 354 and 448, then read the findings register's G5 row and compare the two findings.json and prep.json artefacts from run3-parked. Reproduce the identical-commit comparison if possible; done requires an agreed variance baseline and a defined Box report for inverted axes, rather than treating fixed findings as signal.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai, devtools, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.