Epic: reduce E2E wall time, runner cost, and failure surface

## Problem

NemoClaw's full scheduled E2E workflow takes too long even with broad parallelism, consumes substantial runner capacity, and exposes the run to many independent failure opportunities. Adding more standard runners alone will not address the main bottleneck because recent queue time is low; the cost is dominated by long-running test setup and execution.

A recent full scheduled run took about **29.6 minutes wall-clock**, consumed about **592.5 runner-minutes**, and scheduled **83 non-skipped jobs**. Across five sampled scheduled runs, median wall time was about **32.1 minutes** and median runner consumption was about **595.4 minutes**. All five sampled runs failed, which also indicates an excessive failure surface.

Reference run: https://github.com/NVIDIA/NemoClaw/actions/runs/30503498077

## Scope boundaries with #7614 and #7451

This epic is a follow-on to existing E2E cleanup and feedback-loop work, not a duplicate.

| Owner | Scope |
| --- | --- |
| #7614 | Determine which E2E checks do not protect a unique live boundary; relocate useful assertions into fast projects; retire the corresponding jobs, targets, and matrix rows. |
| #7451 | Reconstruct the specific mixed #7262 diff as focused PRs, including its platform prerequisites and sharding, Hermes production-image reuse, main-workflow scheduling, orchestration timeouts, Brev disposition, protected authorization documentation, and direct PR-gate architecture. |
| #7912 | Improve the runner infrastructure, implementation time, and selective scheduling of the **live E2E journeys retained after #7614 and the completed #7451 slices**. |

The following work is already owned by #7614 and its Priority 2 child #7615, with implementation in #7665, and is explicitly out of scope here:

- Retiring `sandbox-rebuild` and `upgrade-stale-sandbox`.
- Retiring hosted OpenClaw and Hermes inference-switch rows.
- Retiring the Hermes Bedrock row while retaining the canonical OpenClaw seam.
- Moving the associated configuration, classification, output, and state assertions into fast tests.

The following work is already owned by #7451 and is explicitly out of scope here:

- Hosted-runner-loss retry, landed in #7423.
- macOS and WSL platform sharding, landed in #7548.
- Hermes production-image build-once producer/consumer reuse and no-rebuild enforcement, landed in #7508 and #7553.
- Explicit E2E orchestration job timeouts, landed in #7537.
- Starting sandbox image validation after main-workflow preflight, landed in #7552.
- Legacy Brev source-install disposition, owned by #7490 and implemented by #7632.
- Protected internal E2E authorization documentation and direct `E2E / PR Gate` check architecture that remain tracked by #7451.

Guardrails:

1. Do not create a #7912 child issue that reimplements work listed in #7614, #7615, #7665, or #7451.
2. Establish the #7912 baseline only after #7665 is merged or its final retained/removal decisions are reflected.
3. If #7912 analysis finds that a check is not a unique live boundary, route its relocation or retirement back to #7614 rather than duplicating that work here.
4. If a proposal reconstructs or completes a #7262 slice, route it to #7451 rather than duplicating it here.
5. #7912 child issues may optimize or selectively schedule a retained seam, but must name that seam and record why it remains live.

## Goal

Reduce feedback time and runner cost for the retained live E2E suite while preserving every distinct real process, platform, security, migration, and external-system boundary.

Initial measurable targets:

- Nightly E2E p95 wall time: **15–20 minutes** over at least five consecutive runs.
- Nightly median runner consumption: **under 300 runner-minutes**.
- PR-selected E2E: **normally 15 minutes or less**.
- Weekly full compatibility suite: **45 minutes or less**, excluding separate release/Brev validation.

## Workstream 1: infrastructure for retained live journeys

- Activate the existing larger-runner routing as a controlled A/B test. Start with the documented 8-vCPU, 32-GB RAM, 300-GB-disk profile and maximum concurrency of four.
- Route only telemetry-proven CPU-, memory-, or disk-heavy retained jobs to larger runners.
- Do not add standard-runner capacity unless p95 queue time remains above five minutes across several runs. Recent queue time is too low to justify that as the first fix.
- Publish digest-pinned reusable **legacy/versioned** images, plugin fixtures, and exact-commit CLI artifacts so retained jobs do not rebuild identical inputs. This does not reopen the Hermes production-image producer/consumer reuse completed by #7508/#7553.
- Consider ephemeral self-hosted runners only for trusted-main/release workloads if larger hosted runners still show infrastructure loss.
- Keep phase timing, CPU, memory, disk, Docker, queue, retry, and runner-loss telemetry available for comparison.

## Workstream 2: make retained live journeys faster

Prioritize the persistent critical paths that remain after #7665:

- Split the two independent DeepAgents MCP bridge scenarios into parallel jobs or share their setup; target about 12 minutes wall-clock.
- Optimize the retained `openclaw-plugin-runtime-exdev` runtime boundary so it does not perform unnecessary repeated setup; target 15–18 minutes.
- Consume reusable digest-pinned historical OpenClaw and gateway fixtures where doing so preserves the retained live contract.
- Optimize the retained token-rotation transition and no-op paths; any decision to move or retire non-live provider-matrix assertions remains owned by #7614.
- Separate retained functional crash recovery from long soak execution: run a short recovery path nightly and the full repeated-cycle soak weekly.
- Build the NemoClaw CLI once per workflow and distribute a hashed, exact-commit artifact to retained jobs instead of rebuilding it repeatedly.
- Replace fixed sleeps with state-driven readiness checks except when elapsed soak time is itself the contract.

Recent slow paths include `openclaw-plugin-runtime-exdev`, DeepAgents MCP bridge, legacy gateway upgrade, token rotation, `rebuild-openclaw`, channel stop/start, and crash-loop recovery. `upgrade-stale-sandbox` is excluded because its retirement is already owned by #7614/#7665.

## Workstream 3: selectively schedule retained live coverage

Classify the live journeys retained after #7614 into execution tiers:

- **Pull requests:** use changed-area mapping and run only one to three canonical retained live journeys.
- **Nightly:** run roughly 10–15 core retained product journeys.
- **Weekly:** run retained compatibility variants, historical-version coverage, provider variants, and soak tests.
- **Release:** run the exact-commit retained full suite plus staging/Brev launch validation.

Candidate scheduling changes:

- Keep one current retained gateway migration row nightly; move other retained architecture or historical variants weekly.
- Run retained release-baseline plugin compatibility weekly and for releases.
- Keep one representative retained messaging lifecycle nightly; run provider-specific live variants selectively or weekly.
- Run retained issue-specific regression soaks weekly or when their owning files change.
- Support-layer changes should normally run `e2e-support`; map a changed live test file to its canonical retained job instead of broad unrelated E2E groups.

This workstream changes **when retained live coverage runs**. It does not own assertion relocation or test retirement from #7614, and it does not own main-workflow sandbox scheduling or required-check architecture from #7451.

## Proposed child-issue groups

Child issues will be created separately after #7665 is accounted for. Expected groups are:

1. Larger-runner A/B experiment and post-#7665 telemetry baseline.
2. Reusable legacy/versioned images and exact-commit build artifacts for retained jobs, excluding the Hermes production-image reuse completed under #7451.
3. DeepAgents MCP bridge parallelization.
4. Retained OpenClaw plugin EXDEV path optimization.
5. Retained token-rotation path optimization.
6. Recovery-versus-soak scheduling split.
7. Retained gateway and compatibility scheduling tiers.
8. Changed-area dispatch for retained live journeys.
9. Runtime-budget and scheduling policy for retained live E2E.

## Acceptance criteria

- [ ] #7665 is merged or its final retained/removal decisions are reflected before child work begins.
- [ ] No child issue duplicates #7614/#7615/#7665 assertion-relocation or retirement work.
- [ ] No child issue duplicates a #7262 slice or another deliverable owned by #7451.
- [ ] Each child issue names the retained live boundary, baseline, expected savings, owner, and validation plan.
- [ ] Nightly p95 wall time is 20 minutes or less over five consecutive runs.
- [ ] Nightly median runner consumption is under 300 runner-minutes over the same sample.
- [ ] PR-selected E2E normally completes in 15 minutes or less.
- [ ] Unique process, platform, security, migration, and external-system boundaries remain covered.
- [ ] Full compatibility and soak coverage has an explicit weekly or release schedule.
- [ ] Runner-loss failures are distinguishable from product/test failures through telemetry.