ruvnet / ruvnet/RuVector

[PIR][WP12] Build the 30-day continuous-run acceptance test harness

Open
#843 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

phase-3 pir
Dominant language
Rust
Stars
4.5k
Forks
603
Avg merge
23h 32m
Merged PRs (30d)
59

Description

Epic: #837 · Plan: docs/research/perpetual-intelligence-runtime/03-program-plan.md

Goal

Implement the daily-cycle harness that runs the PIR for 30 continuous days and produces a defensible pass/fail verdict on whether the system genuinely improved itself under frozen weights, with full provenance and zero unapproved capability expansion.

Precondition

Foundation-model weights frozen and hash-verified at day 0. WP9's structural gate must confirm no write path to weight files exists for the duration — checked before the clock starts, not after.

Daily cycle

  1. Run the full physical/latent/memory loop over a fixed batch of tasks from a held-out task set never seen during any prior day's mutation testing — otherwise the system overfits the evaluation set and the day-30 numbers mean nothing.
  2. Log per task: latency, task success (binary plus graded where applicable), sensing accuracy (RuView/mmWave ground-truth comparison where available), token/compute cost.
  3. Each day's proposed mutations run: Darwin propose → research-gate evaluate → proof-gate check → constitutional gate if capability-expanding → promote or reject. Every step witness-logged.
  4. Nightly Cloud Scheduler sweeps: causal-audit re-runs (WP6) and witness-chain integrity verification (WP8) against that day's promoted mutations.

Day-30 evaluation criteria

  • Statistical comparison, not mean comparison. Day-30 vs. day-1 per metric (latency, task success, sensing accuracy, cost) via paired bootstrap or equivalent, against a pre-registered significance threshold and minimum effect size. This is the guard against the exact "gains don't prove causation" failure mode that arXiv:2607.26773 (2026-07-29) documents.
  • Full mutation provenance. Every promoted mutation across all 30 days is traceable through the witness chain from proposal → evaluation → promotion, queryable end-to-end in one traversal — which is what the WP8 shared record schema exists to make possible.
  • Zero unapproved capability expansion. Diff the capability/tool/action tables at day 0 vs. day 30. Any expansion without a matching WP11 constitutional-gate approval record fails the acceptance test outright, independent of the performance numbers.
  • Frozen-weights verification. Re-hash foundation-model weights at day 30; must be bit-identical to day 0.
  • An injected-failure rollback test. At least one injected failure during the 30-day window must confirm rollback actually fires live via autogenous's promotion/rollback controller — not merely in isolated unit tests. A rollback path that has only ever passed unit tests is unproven.

Additional acceptance criteria

  • A full 30-day dry run completes and produces a pass/fail report against every acceptance criterion in the brief.
  • Daily checkpoint aggregation runs as a Cloud Scheduler job alongside the existing 7-job pattern.
  • Observability covers all four metric families with retention sufficient to recompute the day-30 statistics from raw logs.
  • Reporting honors the rvm ADR-285 hosted-vs-bare-metal honesty discipline: do not claim verification or isolation strength the deployed topology does not have.

Dependencies

WP2 (#838), WP4 (#840), WP9 (#841), WP10 (#842), WP11 (#851). This is the program's terminal integration point and where every invariant gets an externally-checkable test.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading docs/research/perpetual-intelligence-runtime/03-program-plan.md and the dependency issues WP2, WP4, WP9, WP10, and WP11. Trace the existing Cloud Scheduler 7-job pattern and autogenous's promotion/rollback controller before defining the harness boundaries. Done means a 30-day dry run produces a pass/fail report covering every listed criterion, with recomputable metrics and provenance.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, rust
Domain
ai, cloud, devops, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.