Tracking Issue: build a governed evaluation and improvement loop

Feature description

Define a governed evaluation and improvement loop for retrieval policies, generation profiles, Experience/Skill Candidates, and other context behavior.

The loop should support reproducible experiments and optional sandbox-backed execution while preserving the separation among evidence, validation, human Review, approval, publication, installation, execution, monitoring, and rollback.

Problem and proposed solution

PowerContext already has benchmark infrastructure (#1263), bounded workloads such as #1359, cross-host acceptance (#1358), explainability work (#1356), Candidate Review, and a governed Skill lifecycle (#1397). What is missing is one lifecycle that can answer:

Did this proposed context policy or Skill improve outcomes under a frozen, comparable protocol, and is there enough evidence to publish it?

The RFC should define:

1. an immutable experiment plan with task/dataset selection, baseline, candidate, metrics, seed/order, and environment digest;
2. exact references to the retrieval policy, generation profile, Prompt, Artifact Revision, or Skill package under test;
3. an evaluator/runner SPI that may use the existing E2E/Harbor path or an external sandbox;
4. isolated workspace, filesystem, network, secret, time, token, and cost policies for sandboxed runs;
5. result artifacts containing bounded metrics, outcome categories, exact receipts, and failure provenance;
6. a validation_status independent of Candidate review and publication status;
7. a human decision that may approve, reject, revise, quarantine, or request more evidence;
8. post-publication monitoring through bounded Context Receipts, Task Outcomes, and rollback/quarantine rules.

Sandboxing and evaluation must remain distinct:

• a sandbox limits side effects and state contamination;
• a comparable runner, frozen protocol, metrics, and statistical analysis determine whether the candidate improved outcomes.

PowerContext should integrate a runner rather than become a general Agent runtime.

Acceptance criteria

• A baseline and candidate can run against the same frozen workload with exact version and environment evidence.
• Results distinguish task failure from infrastructure failure, timeout, cancellation, unavailability, and skipped execution.
• A/B or AB/BA ordering, repeated discordant cases, and paired reporting are supported where appropriate.
• The experiment records context bytes/tokens, latency, cost, citation correctness, abstention, and task outcome when applicable.
• A sandboxed run cannot access undeclared secrets, network targets, or persistent state, and its policy is recorded in the result.
• Passing validation does not approve or publish a Candidate; approval does not install or execute it.
• A model or Agent cannot approve its own proposal or silently widen the experiment policy.
• Published behavior can be quarantined or rolled back to an exact prior Revision when monitored evidence crosses a reviewed threshold.
• Partial benchmark runs are labeled as partial and cannot be reported as complete benchmark results.

Alternatives considered

• Treat Review approval as proof of efficacy: Review establishes governance, not measured outcome improvement.
• Treat sandbox success as evaluation success: isolation does not make runs comparable or metrics valid.
• Let the model continuously rewrite and publish itself: collapses evidence, authorization, and rollback boundaries.
• Build a new execution stack per benchmark: duplicates #1263 and makes results difficult to compare.

Additional context

Related: #1263, #1356, #1358, #1359, #1397, and #1391.

• I am willing to contribute code, docs, or design feedback.

No source files or tests are named. Start by reading the benchmark infrastructure in #1263 and the related work in #1356, #1358, #1359, #1397, and #1391, then map the proposed evidence, validation, approval, publication, execution, monitoring, and rollback boundaries. Done means a reviewed RFC that defines the frozen experiment protocol, runner and sandbox responsibilities, result states, governance separation, and rollback rules.