Harness-native mutation certification: require independent review of every code-changing mutation

Problem

Coding agents can successfully remediate one defect while introducing another. The deeper failure is architectural: the same principal that authors a change is often allowed to decide that its own remediation is satisfactory and continue. Tests and self-review help, but they do not create an independent trust boundary.

The invariant I am proposing is:

Every materially code-changing mutation remains untrusted until an independently controlled reviewer certifies the exact resulting bytes. Any remediation after a BLOCK is a new mutation and must traverse the same boundary again.

This is stronger than PR review, self-review, or action authorization. It is mutation certification.

Desired harness property

Conceptually:

author agent
    |
    v
candidate filesystem mutation
    |
    v
isolated/untrusted candidate state
    |
    +--> deterministic evidence (tests/static/type/build as applicable)
    |
    v
independent adversarial reviewer
    |
    +-- BLOCK --> author remediates --> NEW candidate --> NEW review
    |
    +-- CLEAR --> clearance bound to exact resulting content digest
                     |
                     v
               promote/trust state

The authoring principal should not be able to bypass, forge, inherit, or self-certify the clearance. Editing even one covered byte invalidates the previous clearance. A fix prompted by the reviewer receives no inherited trust from the previous review.

Why harness-native matters

I built a userland implementation around git and an external reviewer. It uses:

• an independent external model reviewing the exact staged change;
• CLEAR/BLOCK verdicts tied to staged blob SHA-256s;
• fresh review after every remediation;
• pre-commit enforcement;
• post-commit notarization into refs/notes/adversary;
• a history auditor that recomputes committed blob hashes;
• pre-push enforcement and propagation of the evidence notes;
• CI re-audit and protected-branch enforcement;
• a harness-side deny guard intended to raise the cost of agent bypass/forgery.

The important limitation discovered in production is that userland cannot close the final bypass class cleanly. A sufficiently capable agent can potentially reach around client-side mechanisms. The natural enforcement point is the harness/tool boundary itself, where mutation can be made conditional on independent certification.

Production evidence

The implementation maintains an append-only evidence docket recording catches that forced remediation, including catches in the gate itself. The current docket contains 35 evidence events. Particularly relevant examples include:

• EV-016: remediation for an archive/schema defect itself silently discarded two real review records. Reviewing the remediation like any other mutation stopped it from landing.
• EV-018: promotion of a new reviewer model was itself BLOCKed multiple times by that reviewer, exposing a reachable crash, a dead fallback, and false invariant claims before CLEAR.
• EV-020: the gate caught confident-wrong claims in the author's own gated review deliverables (a dropped finding and an incorrect count).
• EV-022: hardening the harness guard took eight review rounds; successive rounds found agent-reachable forgery/bypass routes and integrity defects in the security mechanism itself.
• EV-028 / EV-030: repeated adversarial review of path-canonicalization hardening found additional Windows/path ambiguity bypasses after earlier rounds had already cleared related implementations. A port reviewed in a different context found defects missed in the original.
• EV-029: the reviewer returned CLEAR while an .mjs security guard was accidentally excluded from the code-extension set. The reviewer's own scope statement exposed that the supposedly reviewed security file had not actually entered the payload.
• EV-031: a later full-file reread found a diff-generation blind spot in previously reviewed gate code (color.diff / external diff behavior). This reinforced that “reviewed” is contextual, not an eternal property of bytes.
• EV-033–035: byte-identical moved code exposed long-lived dead-on-arrival defects; subsequent evidence/runbook changes were themselves caught making unsupported claims; the first real history rewrite exposed live-only flaws that rehearsal had not modeled.

Earlier production measurements also showed roughly one in four gated code-bearing commits in the initial observed period carried a real defect that the adversarial gate forced out before shipping. I would not claim that ratio is universal; it is evidence of the failure mode, not a general benchmark.

The strongest recurring observation is not merely that agents make mistakes. It is that remediation is another mutation with the same epistemic status as the original change, and authors can confidently defend incorrect claims about changes they just made. Therefore self-review alone cannot provide the desired trust boundary.

Reference implementation / evidence

Design and evidence documentation:

https://github.com/Nexusmill/colibri-code-review/tree/main/docs

Relevant implementation:

https://github.com/Nexusmill/Tools/tree/main/adversary-gate

Public/plugin distribution:

https://github.com/Nexusmill/colibri-marketplace/tree/main/plugins/adversary-gate

The documentation includes ADVERSARY_GATE.md, LAYERED_ENFORCEMENT.md, GATE_ADOPTION_PLAYBOOK.md, HARNESS_GUARD.md, and the running GATE_EVIDENCE_DOCKET.md / gate_evidence.json.

Request

Please consider making independent mutation certification a harness-level primitive in Codex rather than an optional agent workflow:

1. Treat code-changing tool results as candidate/untrusted state.
2. Invoke an independently controlled reviewer before promotion to trusted state.
3. Bind CLEAR to the exact resulting content/state digest.
4. Invalidate CLEAR on any subsequent covered mutation.
5. Treat reviewer-requested remediation as a fresh mutation requiring a fresh review.
6. Prevent the authoring principal from mutating/forging reviewer state or bypassing the certification boundary.
7. Preserve auditable evidence of what bytes/state were reviewed and what reviewer/verdict authorized promotion.
8. Fail closed when the independent reviewer cannot produce a valid verdict, with an explicit human override path where appropriate.

The underlying model can improve indefinitely, but this addresses a different problem: who is allowed to certify the model's own changes?

I believe the appropriate answer is: not the authoring principal itself.

Start with the linked design documents, especially ADVERSARY_GATE.md, LAYERED_ENFORCEMENT.md, and HARNESS_GUARD.md, then inspect the referenced adversary-gate implementation. Map the proposal onto Codex's mutation and promotion boundaries before deciding where a harness-level primitive could live. Done means covered mutations require independent, digest-bound certification, remediation triggers fresh review, bypasses are rejected, and evidence is auditable.