appandflow / appandflow/stim

Reduce duplicate release QA work with coverage-aware orchestration

Aperta
#598 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
TypeScript
Stelle
41
Fork
1
Merge medio
1h 25m
PR unite (30g)
397

Descrizione

## Problem

Stable-release QA repeats expensive native preparation, cold builds, warm launches, and cleanup across template lifecycle/cache suites and real-repository manual/lifecycle/cache passes. This makes the release's critical path unnecessarily long and hard to track. The 1.0.0 release retains its existing checklist; the release owner requested this follow-up for implementation after publication.

## Evidence

- `RELEASE.md` section 3 requires the full major-release matrix on two representative real repositories. `docs/field-test-protocol.md` also requires manual launch/error/recovery and cache evidence.
- In 1.0.0 QA, the real Expo iOS lifecycle and cache suites took approximately 10 and 20 minutes; Android took approximately 7 and 20 minutes. Real bare iOS took approximately 5 and 10 minutes, in addition to native preparation and manual interaction.
- Both lifecycle and cache suites independently exercise cold launch, cross-worktree artifact reuse, and cleanup. Manual verification adds another warm launch, although UI interaction and deliberate error/recovery are distinct valuable checks.
- The campaign also found genuine correctness/remedy issues (#576, #578, #589, #596) and a needless populated-Metro-cache wait (#583). Reducing QA must preserve the assertions that caught those failures.
- Heavy native work runs sequentially on the 16 GB QA host. Reviews, report assembly, packaging inspection, and remote CI need not all sit on that same critical path.

## Cause

The runners own separate fixtures and repeat overlapping setup. Release orchestration lacks a shared coverage/evidence manifest, a dependency-aware schedule, and a clear policy for retaining unaffected evidence after a narrow candidate fix. Preparation/tooling friction and real regressions also consume time; not all elapsed time is removable test duplication.

## Fix idea

- Combine compatible lifecycle and cache execution so one cold/warm pair satisfies both sets of assertions; add manual UI/error/recovery at the appropriate lifecycle stage.
- Preserve deliberate fresh-cache boundaries, single-flight races, final-key correctness, cache engagement/storage/reuse, launch verification, process/device ownership, and all cleanup assertions. Never let setup preseed a cache that a test claims is cold.
- Define which candidate changes invalidate which evidence. Rerun affected rows after a fix, retain unaffected results with their exact commit and a documented applicability reason, and keep final packaging/version/exact-commit CI gates.
- Schedule low-impact review/report/package inspection and remote CI in parallel where safe. Keep benchmark measurements isolated and heavy native builds sequential on this host unless resource measurements justify another policy.
- Produce a concise current-state report plus stage timings; distinguish setup, active test time, regressions, retries, and review/CI wait. Preserve private failed-attempt evidence.
- Update release documentation and actual orchestration together. Validate equivalent coverage and measure total critical-path reduction on representative Expo and bare apps, including a seeded failure that must still fail.

Do not shorten the active 1.0.0 checklist as part of this ticket. Claim and implement after that release is published.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.