Nimbus: build hermetic coding and instrumentation calibration corpus

Parent: #17

Depends on the offline scaffold, correctness oracle, and accepted smevals/OpenCode boundary spike.

Outcome

Build the fictional, hermetic Nimbus corpus used only for:

• executor/instrumentation calibration;
• isolation and deterministic-scorer checks;
• mechanical artifact/MCP/access-counter probes; and
• baseline-only OpenCode model/provider qualification.

Nimbus provides no treatment-effect estimate and no external-validity evidence. It must never select a model based on matlatl responsiveness.

Scope

• A coherent fictional cloud-native product with natural Markdown documentation and source code/test infrastructure.
• Coding fixtures whose correct patches depend on documented behavioral, security, configuration, or architecture constraints.
• Disposable qualification tasks covering the four primary coding strata: grep-friendly control, cross-document synthesis, navigation-heavy coding, and single-document constraint.
• Frozen documentation topology and independently derived graph expectations where needed by the mechanical probes.
• Private deterministic verifiers, gold isolation, exact reset/cleanup, and immutable content hashes.
• Scripted probes proving:
  • baseline removes pre-existing matlatl artifacts and MCP entries;
  • +all (machine ID all) injects exact llms.txt, trails.json, availability notice, and remote streamable-HTTP /mcp configuration;
  • llms.txt and trails.json access counters increment exactly;
  • every matlatl MCP tool counter increments and reconciles with the trajectory;
  • only explicit cache read/write 0/0 is valid;
  • token, billed-cost, turn, tool, and wall-time fields map exactly;
  • known correct/incorrect patches score deterministically; and
  • gold and host evaluator state remain inaccessible to the agent container.

Constraints

• Follow docs/research/nimbus-eval-corpus.md.
• Nimbus is separate from demo/corpus/nimbus-docs; do not modify or reuse the demo.
• No runtime network dependency for corpus/scorer mechanics.
• Qualification uses baseline only. Do not generate +all, compute a treatment delta, or reuse qualification tasks in real-corpus endpoints.
• Directional arm behavior is not a Nimbus acceptance criterion.
• task dogfood must remain green.

This body supersedes the earlier controlled-treatment-discrimination and documentation-only coverage wording.

Acceptance

• Documentation, source code, tests, and exact corpus manifest are frozen and byte-stable.
• Every coding fixture has a private deterministic verifier and documented-constraint mapping reviewed independently.
• Task-family and primary-stratum assignments are frozen; qualification tasks are explicitly disposable.
• Every scripted artifact, MCP, access-counter, zero-cache, telemetry, scorer, reset, and isolation probe passes exactly.
• Construction expectations are hand-enumerated or independently derived rather than copied from matlatl output.
• Gold/verifier material is absent and inaccessible from the agent container.
• Baseline-only qualification manifest supports frozen competence, protocol reliability, telemetry completeness, projected-cost, and deterministic tie-break criteria.
• No Nimbus result is reported as a treatment effect or evidence about real repositories.
• task dogfood remains green.