NVIDIA / NVIDIA/NemoClaw

ci(e2e): enforce runtime budgets for retained live E2E

Open
#7,922 0 comments 0 reactions 1 assignee Claimed by @cjagwani View on GitHub
area: e2e area: performance
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

Parent: #7912

## Problem

Live E2E tests currently lack one enforceable admission and scheduling contract. Without a runtime budget, unique-boundary statement, selective mapping, and review point, the suite can grow back into a slow and noisy default matrix.

## Scope

- Define required metadata for new retained live E2E coverage:
- unique live boundary;
- expected runtime and runner class;
- PR/nightly/weekly/release tier;
- owning-file mapping;
- required telemetry and artifacts; and
- review, consolidation, or retirement condition.
- Add a machine-checkable policy or repository validation where practical.
- Document how exceptions are approved and measured.
- Base initial budgets on the post-#7665 and post-optimization baseline.

## Ownership boundaries

- This issue governs retained live coverage; it does not decide #7614 retirements.
- Do not duplicate #7451 required-check architecture, #7145 runner activation, or #7146 telemetry implementation.

## Acceptance criteria

- [ ] New retained live targets cannot be added without the required ownership and runtime metadata.
- [ ] Validation distinguishes policy failure from a live-test failure.
- [ ] Existing retained targets have an initial tier and budget.
- [ ] Exceptions require explicit rationale and an expiry or review condition.
- [ ] The policy supports the #7912 wall-time and runner-minute targets.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.