[PIR][WP12] Build the 30-day continuous-run acceptance test harness
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 4.5k
- Forks
- 603
- Avg merge
- 23h 32m
- Merged PRs (30d)
- 59
Description
Epic: #837 · Plan: docs/research/perpetual-intelligence-runtime/03-program-plan.md
Goal
Implement the daily-cycle harness that runs the PIR for 30 continuous days and produces a defensible pass/fail verdict on whether the system genuinely improved itself under frozen weights, with full provenance and zero unapproved capability expansion.
Precondition
Foundation-model weights frozen and hash-verified at day 0. WP9's structural gate must confirm no write path to weight files exists for the duration — checked before the clock starts, not after.
Daily cycle
- Run the full physical/latent/memory loop over a fixed batch of tasks from a held-out task set never seen during any prior day's mutation testing — otherwise the system overfits the evaluation set and the day-30 numbers mean nothing.
- Log per task: latency, task success (binary plus graded where applicable), sensing accuracy (RuView/mmWave ground-truth comparison where available), token/compute cost.
- Each day's proposed mutations run: Darwin propose →
research-gateevaluate → proof-gate check → constitutional gate if capability-expanding → promote or reject. Every step witness-logged. - Nightly Cloud Scheduler sweeps: causal-audit re-runs (WP6) and witness-chain integrity verification (WP8) against that day's promoted mutations.
Day-30 evaluation criteria
- Statistical comparison, not mean comparison. Day-30 vs. day-1 per metric (latency, task success, sensing accuracy, cost) via paired bootstrap or equivalent, against a pre-registered significance threshold and minimum effect size. This is the guard against the exact "gains don't prove causation" failure mode that arXiv:2607.26773 (2026-07-29) documents.
- Full mutation provenance. Every promoted mutation across all 30 days is traceable through the witness chain from proposal → evaluation → promotion, queryable end-to-end in one traversal — which is what the WP8 shared record schema exists to make possible.
- Zero unapproved capability expansion. Diff the capability/tool/action tables at day 0 vs. day 30. Any expansion without a matching WP11 constitutional-gate approval record fails the acceptance test outright, independent of the performance numbers.
- Frozen-weights verification. Re-hash foundation-model weights at day 30; must be bit-identical to day 0.
- An injected-failure rollback test. At least one injected failure during the 30-day window must confirm rollback actually fires live via autogenous's promotion/rollback controller — not merely in isolated unit tests. A rollback path that has only ever passed unit tests is unproven.
Additional acceptance criteria
- A full 30-day dry run completes and produces a pass/fail report against every acceptance criterion in the brief.
- Daily checkpoint aggregation runs as a Cloud Scheduler job alongside the existing 7-job pattern.
- Observability covers all four metric families with retention sufficient to recompute the day-30 statistics from raw logs.
- Reporting honors the
rvmADR-285 hosted-vs-bare-metal honesty discipline: do not claim verification or isolation strength the deployed topology does not have.
Dependencies
WP2 (#838), WP4 (#840), WP9 (#841), WP10 (#842), WP11 (#851). This is the program's terminal integration point and where every invariant gets an externally-checkable test.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading docs/research/perpetual-intelligence-runtime/03-program-plan.md and the dependency issues WP2, WP4, WP9, WP10, and WP11. Trace the existing Cloud Scheduler 7-job pattern and autogenous's promotion/rollback controller before defining the harness boundaries. Done means a 30-day dry run produces a pass/fail report covering every listed criterion, with recomputable metrics and provenance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, rust
- Domain
- ai, cloud, devops, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100