unic / unic/unic-agents-plugins
spike(unic-archon-dlc): the review Box is not deterministic across iterations on identical code
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 19
Description
Observed
Run 3, two unic-dlc-pr-review iterations on PR 5881 at the same commit 08e2523. Iteration 1: 14 findings (Standards 9, Spec 5), deduped to 13. Iteration 2: 13 findings (Standards 3, Spec 10). 0 of 13 prior hashes reappeared. By hand: 10 are the same defects reworded, re-anchored or axis-flipped, 3 are new (all Spec, all acceptance-criteria reads iteration 1 did not do), 6 vanished (all Standards). Praise: 13 bullets each; about 7 survive reworded, 3 file-and-line bullets vanished, one new bullet praises what iteration 1 filed as a defect.
The hash includes the normalised title (unic-dlc-pr-review.yaml:354), so rewording alone explains the 0/13; reconcile's semantic fallback (:448) matched 7. Run 2 reproduced six hashes, which now reads as less rewording, not determinism.
Why it matters
The register's run-2 comparables (dedup, reconcile, praise) assume a stable reviewer. Run 3 measured the variance once; run 4's seal needs a baseline before any fixed is read as signal (#{G4}).
Evidence
- Findings register, § "G. What run 3 taught about the instrument", row G5: https://github.com/unic/unic-agents-plugins/blob/bcf688a/docs/research/2026-08-31-dlc-run-2-findings.md
- Run record and harvested artefacts:
~/Desktop/run3-parked/on the maintainer's machine (artefacts-26f0f16c…/findings/review.json,artefacts-908e3735…/findings/review.json, bothprep.json) - Dispatch record: #457 from comment 5698289748 down
Not decided
- How many identical-code iterations make a variance baseline, and what the Box should report when the axis split inverts.
- Whether the Standards axis should be seeded with the prior iteration's findings as things to confirm or retract, rather than re-derived blind.
Filed 2026-09-17 by wayfinder-orchestrator-10 from run 3's measurements. needs-specs: grill before dispatch.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with unic-dlc-pr-review.yaml at lines 354 and 448, then read the findings register's G5 row and compare the two findings.json and prep.json artefacts from run3-parked. Reproduce the identical-commit comparison if possible; done requires an agreed variance baseline and a defined Box report for inverted axes, rather than treating fixed findings as signal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- ai, devtools, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100