NVIDIA / NVIDIA/NemoClaw

Retire or consolidate six redundant full-matrix E2E jobs

Open
#11,946 1 comment 0 reactions 1 assignee Claimed by @rsliter View on GitHub
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Problem

Full-matrix main run [35118412060](https://github.com/NVIDIA/NemoClaw/actions/runs/35118412060) reported 29 primary failed jobs. Six of those jobs no longer justify an independent full-matrix execution slot after recent product-ownership decisions and coverage consolidation.

Failures are not themselves a reason to delete coverage. Each job below is included because its product contract has moved to a native owner, is already proved by another retained live boundary, or is deterministic and belongs in a smaller test lane.

## Product Decision and Outcome

**Decision: Accept.**

- **Reason:** remove six redundant or obsolete full-matrix jobs while preserving every supported observable, recovery, cleanup, and security outcome at its current authoritative boundary.
- **Placement:** NemoClaw E2E workflow, target catalogue, live fixtures, deterministic tests, and owning E2E documentation.
- **Accountable owner:** NemoClaw E2E maintainers, coordinated with the owners of #11766 and #11905.
- **Validation:** exact-head deterministic owners, targeted retained live lanes, and one completed full manual main-equivalent run must prove the reduced matrix before the final deletion merges.

## Jobs and Required Dispositions

| Current job or target | Disposition | Required retained owner | Existing authority or overlap |
| --- | --- | --- | --- |
| `openclaw-plugin-runtime-exdev` | Delete the dedicated job, target, live fixture, and support contracts. | `full-e2e`, `rebuild-openclaw`, `rebuild-hermes`, managed-image activation, and messaging-provider native lifecycle proofs. | Accepted native plugin/package ownership in #11766; implementation in #11906. |
| `ubuntu-repo-docker-post-reboot-recovery` | Delete the current direct-Docker simulated-reboot target. Preserve only supported state survival through the retained OpenShell lifecycle owner. | `openshell-stop-start-lifecycle` plus deterministic exact-identity, cleanup, and state-preservation tests. | Accepted OpenShell lifecycle ownership in #11905; implementation coordination with #11909. |
| `issue-2478-crash-loop-recovery` | Delete the dedicated NemoClaw recovery job. If native self-recovery remains a supported qualification outcome, retain one bounded observation in an agent-owned qualification lane without NemoClaw repair authority. | `full-e2e` or the applicable native-agent qualification owner, plus deterministic NemoClaw no-intervention coverage. | Native agents own gateway and background-process lifecycle under #11905 and its prerequisite ownership decisions. |
| `Shared E2E (onboard-managed-image-buildless-e2e-docker)` | Remove this deterministic integration test from full live-matrix dispatch. Do not delete its useful deterministic coverage. | Ordinary CI execution of `test/onboarding/onboard-managed-image-buildless-e2e.test.ts`. | The failed assertion was stale documentation prose and exercised no real sandbox, network, process, or service boundary. |
| `device-auth-health` | Delete the standalone live job after moving or confirming all 401/status classification behavior at deterministic owners. Retain one real dashboard observation only if it is not already covered. | `src/lib/verify-deployment.test.ts`, gateway HTTP readiness/reuse tests, status health tests, and `dashboard-remote-bind` for the remaining live dashboard boundary. | Existing deterministic 401 coverage and overlapping dashboard plus gateway-recovery E2Es. |
| `cloud-inference` | Delete the standalone job after disposing of its unique assertions. | `full-e2e` and `bootstrap-install-smoke` for install/onboard/inference; security tests for credential isolation; deterministic image/skill contracts for filesystem shape. | The live install, PATH, onboard, and hosted-inference outcomes already have retained live owners. Repo skill validation and exact filesystem shape do not require a live provider. |

## Scope

### Included

- Remove the six independent workflow jobs or dispatch entries listed above.
- Delete job-specific live fixtures and support contracts that have no remaining consumer.
- Move deterministic behavior to the smallest existing unit, integration, package-contract, or `e2e-support` owner before deleting live assertions.
- Consolidate unique live behavior into an existing retained target only when it represents a distinct supported external boundary.
- Update the workflow planner, target catalogue, mock-parity data, assertion census, selector validation, documentation, and report aggregation.
- Record before-and-after target, job, live-test, assertion, runtime, and line counts.

### Excluded

- Do not retire either historical upgrade target from run 35118412060. The v0.0.115 Shields retirement migration was intentionally retained after Shields left core, and the v0.0.89 target remains an old-to-current compatibility boundary.
- Do not retire snapshot/restore. Broader snapshot removal is explicitly outside #11905.
- Do not retire `openshell-stop-start-lifecycle`; it is required validation for #11905.
- Do not retire OpenClaw or Hermes security-posture qualification.
- Do not remove MCP, Pi, GPU Ollama, Model Router, Brave Search, provider switching, staging Launchable, performance, CLI routing, bootstrap installation, dashboard remote-bind, or protected managed-image GPU coverage under this issue.
- Do not weaken fail-closed cleanup, exact ownership, secret redaction, negative policy, or credential-isolation evidence.
- Do not add replacement jobs, registries, compatibility layers, broad retries, or new matrix permutations.

## Coverage-Transfer Requirements

For every removed live assertion, record exactly one disposition:

1. already covered by a named retained deterministic or live test;
2. moved to a named smaller test in the same change;
3. covered by another retained observable behavior test; or
4. no quality value because it checks incidental output, internal representation, or third-party behavior.

Do not move assertions into helpers, snapshots, aggregate receipts, shell probes, or opaque composite checks merely to reduce the visible count.

## Acceptance Criteria

- [ ] The six listed jobs no longer execute in a full manual E2E run.
- [ ] `openclaw-plugin-runtime-exdev` removal is delivered through or reconciled with #11906 and closes or updates the conflicting retention record in #11547.
- [ ] Post-reboot and crash-loop removal is delivered through or reconciled with the final E2E inventory required by #11905 and #11909.
- [ ] The buildless onboarding test remains in ordinary deterministic CI and is no longer reported as a live full-matrix E2E job.
- [ ] HTTP 401 means reachable at every supported status/readiness boundary, proved by focused deterministic tests.
- [ ] Dashboard remote-bind retains its distinct real dashboard exposure and audit outcome without inheriting unrelated recovery assertions.
- [ ] Full E2E or another retained live owner proves install, onboarding, sandbox inference, and the relevant credential boundary formerly repeated by `cloud-inference`.
- [ ] Every deleted assertion has a reviewed coverage disposition and every moved behavior has fail-before/pass-after evidence at its new owner.
- [ ] Negative, recovery, cleanup, exact-owner, redaction, and security outcomes remain covered at their authoritative boundaries.
- [ ] Manual selectors for deleted targets are either rejected with the existing unknown-selector contract or handled by an explicitly time-bounded compatibility rule. No silent alias is added.
- [ ] Workflow planner, target catalogue, mock parity, semantic phase checks, assertion census, documentation, and report aggregation pass with no stale references.
- [ ] The completed change is net-negative in workflow, test, and support lines and introduces no replacement lifecycle or test infrastructure.
- [ ] Targeted retained live lanes pass on the exact final commit.
- [ ] One completed full manual E2E run on the exact final commit confirms the reduced matrix and reports no missing aggregate dependency.

## Security and Failure Boundaries

- Credential locations, access, lifetime, and removal do not change under this issue.
- Credential-isolation and redaction assertions removed from `cloud-inference` must have named retained owners before deletion.
- A failed or ambiguous cleanup remains a failure. Job consolidation must not replace exact identity and absence checks with name-only or best-effort cleanup.
- Device-authenticated HTTP 401 responses remain classified as reachable without exposing authentication material.
- Native gateway failure must not trigger reintroduction of a NemoClaw-owned process or container recovery controller.
- Workflow and external-resource writes remain restricted to existing trusted E2E boundaries. This issue adds no new credentials, permissions, or external resources.

## Delivery Order and Dependencies

1. Land or reconcile #11906, including replacement native plugin lifecycle evidence and #11547 disposition.
2. Complete the lifecycle-owned inventory in #11909 for the post-reboot and crash-loop jobs.
3. Relocate deterministic buildless and device-auth coverage.
4. Consolidate the remaining unique `cloud-inference` evidence into named retained owners.
5. Remove stale selectors, workflow aggregation edges, fixtures, and documentation together with the final jobs.
6. Run targeted evidence and one exact-head full manual E2E.

Stop rather than delete a job if discovery finds a unique supported external behavior without a retained owner, if #11906 or #11909 changes its accepted ownership boundary, or if the exact-head full run cannot prove the reduced aggregate.

## Related Work

- #11766 and #11906: native plugin and package lifecycle ownership; explicit EXDEV job retirement.
- #11905 and #11909: retirement of NemoClaw-owned Docker and Podman lifecycle orchestration.
- #11547: prior decision to retain the EXDEV live boundary; must be reconciled with the newer accepted native-ownership decision.
- #10517: completed high-cost E2E pruning, including intentional retention of the Shields retirement-upgrade boundary.
- #10920: assertion pushdown rules. It does not itself authorize target or job deletion.
- #10722: Shields product removal and retained legacy-state migration boundary.
- #11775: OpenShell capability-enforcement ownership with intentionally retained OpenClaw and Hermes security-posture E2Es.

## Checklist

- [x] The requested deletion set is bounded to six named jobs.
- [x] Each proposed deletion names a retained owner or a required ownership decision.
- [x] Conflicting and dependent issues and PRs are identified.
- [x] Security, cleanup, recovery, and credential evidence are preserved.
- [ ] Assign one implementation owner after overlapping PRs establish their final scope.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.