[P1] Pause the benchmarks that cannot produce trustworthy data, and re-scope the measurement approach

**Parent Epic:** [#6944 Performance Quality Gates](https://github.com/MetaMask/MetaMask-planning/issues/6944) → [#7182 Benchmarks: Reliability](https://github.com/MetaMask/MetaMask-planning/issues/7182)
**Size:** M | **Hours:** ~8-12

---

## Decision

The team has aligned on pausing the benchmarks that cannot currently produce trustworthy data — `startupPowerUserHome` and the CUF flows — and on reconsidering the measurement approach before restoring them.

This ticket records what that removes, what it leaves running, and what has to be true before anything comes back.

**No single defect motivates this and no single fix reverses it.** The flows below each carry several independent faults, filed separately over the last five months, and a benchmark is only worth running again when all of the ones affecting it are closed.

## `startupPowerUserHome`

| what is wrong | where |
|---|---|
| exits 0 having emitted zero metrics in **76 of 85** `main` runs | [extension#45945](https://github.com/MetaMask/metamask-extension/issues/45945) |
| CV 30-34% when it does emit — the reason `CI_MULTIPLIER.STARTUP_POWER_USER` is 2.0, the widest band in the table | `thresholds.ts` |
| the flow carries no gated timer at all | `gated-metrics.ts` |

Its instrument is sound: it shares `driver.collectMetrics()` with `startupStandardHome`, so the difference between the two is fixture and workload, not measurement. Pausing costs close to nothing, because it is already dark and paying runner time to stay that way.

## The CUF flows

| what is wrong | where |
|---|---|
| step timers run their clock in the Node test process, so app work, WebDriver round-trip and poll interval are inseparable; a wait-terminated step cannot resolve finer than 200 ms | [extension#46006](https://github.com/MetaMask/metamask-extension/issues/46006) |
| step timers are not a disjoint partition; a named step excludes its own trigger, and adjacent labels trade a shared wait | [extension#45452](https://github.com/MetaMask/metamask-extension/issues/45452) |
| `total` is the sum of the step timers, so every trigger and inter-step gap is in no metric by construction | [extension#45452](https://github.com/MetaMask/metamask-extension/issues/45452) |
| a step recording zero is dropped from both the step list and the sum, so a broken timer makes `total` *smaller* with no trace | [extension#45452](https://github.com/MetaMask/metamask-extension/issues/45452) |
| onboarding metrics are bimodal — a ~37% per-iteration slow path, so the value is a coin flip rather than a measurement of one thing | [extension#45266](https://github.com/MetaMask/metamask-extension/issues/45266) |
| password-transition timers wrap an instant transition, measuring a near-empty window | [#7280](https://github.com/MetaMask/MetaMask-planning/issues/7280) |
| mean, stdDev and percentiles averaged independently across different run sets | [#7204](https://github.com/MetaMask/MetaMask-planning/issues/7204) |
| sub-50 ms steps gated below the detection floor; one is dropped for `cv > 50` in 12 of 116 runs while still being gated | [#7281](https://github.com/MetaMask/MetaMask-planning/issues/7281) |
| an iteration that fails or stalls emits nothing | [extension#45431](https://github.com/MetaMask/metamask-extension/issues/45431) |
| network stub coverage, deterministic render-complete waits, GC normalization — all open | [#7201](https://github.com/MetaMask/MetaMask-planning/issues/7201), [#7202](https://github.com/MetaMask/MetaMask-planning/issues/7202), [#7203](https://github.com/MetaMask/MetaMask-planning/issues/7203) |
| the 2026-08-24 mock pivot moved levels, so any trend crossing it compares two different quantities | [extension#45679](https://github.com/MetaMask/metamask-extension/pull/45679) |
| ceilings were derived from production Sentry percentiles — a third population, gating neither CI measure | [extension#45205](https://github.com/MetaMask/metamask-extension/issues/45205), [#7252](https://github.com/MetaMask/MetaMask-planning/issues/7252) |

Several of these are independent of each other. Deterministic waits do not fix the clock; the clock does not fix bimodality; none of them fixes a ceiling derived from the wrong population.

## What this removes from the gate

Of the 20 entries in `GATED_METRIC_VALUES`, **9 of the 12 gated timing metrics are CUF-derived** — six step timers plus `onboardingImportWallet.total`, `importSrpHome.total` and `swap.total`, which are sums of those steps.

`onboardingImportWallet.total` is among them, and it is the only metric that has ever blocked a pull request: 27 of 116 Chrome runs on `main`, 2026-08-27 to 09-03. **So the gate stops blocking on timing entirely.** That is the honest cost and it belongs in the PR that lands this, not in a later discovery.

The other eleven were not blocking anything. They sit 1.4x to 147x under their ceilings and would need regressions of that order to fire.

## What keeps running

| | |
|---|---|
| 8 CLS canaries | unitless, `CI_MULTIPLIER.NONE`, an absolute null of near zero rather than a calibrated ceiling. The only gated family that has produced signal |
| `startupStandardHome.load` | `navigationEntry.loadEventEnd`, read in-page, from [#7869 Add benchmark script](https://github.com/MetaMask/metamask-extension/pull/7869) |
| `startupStandardHome.loadScripts`, `uiStartup` | in-page via `stateHooks.getCustomTraces()`, from [#27701 include custom traces in benchmark results](https://github.com/MetaMask/metamask-extension/pull/27701) |

These are not clean, they are *cleaner*: their ceilings are still 2.0x to 5.5x above the observed distribution, because every base was calibrated against local timings and scaled by a fixed tier multiplier rather than derived from the population it gates. They keep running as measurements and are not load-bearing as gates until [extension#45205](https://github.com/MetaMask/metamask-extension/issues/45205) re-derives them.

## Reconsidering the approach — what has to be settled

1. **Where the clock lives for a CUF step.** [extension#46006](https://github.com/MetaMask/metamask-extension/issues/46006) proposes moving it in-page via the mechanism `loadScripts` already uses. Needs a worked example on one step before the rest follow.
2. **Whether a flow gets a flow-level span.** Without one, `total` is the sum of its steps identically, always, so no coverage gap is detectable. With one, `total` minus the sum is a real residual to assert on.
3. **What sample size a comparison actually has.** Within-run iterations are not between-run samples: on `onboardingImportWallet.total`, within-run sd is 120 ms against 862 ms between-run. Anything built on iterations inside a single CI run has no confidence interval on its delta.
4. **Whether ceilings come from the population they gate**, and whether a metric whose minimum detectable effect exceeds its usefulness should be gated at all.
5. **What restores a paused benchmark** — a condition per flow, stated as a checkable property and naming every ticket above that bears on it, rather than a date.

## Acceptance Criteria

- [ ] `startupPowerUserHome` no longer runs on the per-commit path
- [ ] The CUF flows no longer run on the per-commit path, or run report-only with no gated entries
- [ ] Every remaining entry in `GATED_METRIC_VALUES` has a named reason it is trustworthy, and the reason is checkable
- [ ] The gate's loss of every blocking timing metric is stated in the PR that lands this
- [ ] Each paused benchmark carries a restore condition listing every open ticket that bears on it
- [ ] Runner time released by the pause is measured and reported

## Labels

`team-extension-platform`, `area-testSuite`

## Dependencies

Related: [extension#46006](https://github.com/MetaMask/metamask-extension/issues/46006), [extension#45452](https://github.com/MetaMask/metamask-extension/issues/45452), [extension#45266](https://github.com/MetaMask/metamask-extension/issues/45266), [extension#45945](https://github.com/MetaMask/metamask-extension/issues/45945), [extension#45431](https://github.com/MetaMask/metamask-extension/issues/45431), [extension#45205](https://github.com/MetaMask/metamask-extension/issues/45205), [extension#45935](https://github.com/MetaMask/metamask-extension/issues/45935), [#7280](https://github.com/MetaMask/MetaMask-planning/issues/7280), [#7204](https://github.com/MetaMask/MetaMask-planning/issues/7204), [#7281](https://github.com/MetaMask/MetaMask-planning/issues/7281), [#7201](https://github.com/MetaMask/MetaMask-planning/issues/7201), [#7202](https://github.com/MetaMask/MetaMask-planning/issues/7202), [#7203](https://github.com/MetaMask/MetaMask-planning/issues/7203), [#7252](https://github.com/MetaMask/MetaMask-planning/issues/7252)

Start by reading thresholds.ts and gated-metrics.ts, then trace how the benchmark flows and GATED_METRIC_VALUES are selected for the per-commit path. Review the linked reliability issues before deciding the pause and restore conditions. Done means the named flows are paused or report-only, remaining gated entries have checkable trust reasons, timing-gate loss is documented, and runner-time savings are measured.

Written by the indexing model from the issue text.