aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

perf(ci): shard the CDK jest suite (jest --shard) — DEFERRED, follow-up to #363

Aperta
#675 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
ci-cd enhancement
Lingua principale
TypeScript
Stelle
143
Fork
46
Merge medio
3g 10h
PR unite (30g)
24

Descrizione

Parent: #363 (build-performance umbrella). Spun from #363's DoD by the resolve-open-issues orchestrator; **needs maintainer `approved` before implementation (ADR-003)**.

> **⚠️ Re-framed 2026-07-29 after measured review of PR #364 (@theagenticguy).** This lever was originally filed as "the biggest remaining win." **Measurement shows it is currently net-neutral-to-slower and should be DEFERRED behind cheaper levers.** The design below is kept because it is the part people get wrong; the *priority* is what changed.

## Status: DEFERRED (documented design, do not implement yet)

## Why deferred — the numbers moved
The original ~298s `//cdk:test` figure was stale. After PR #371 (disable Lambda bundling in CDK unit-test synths) the suite is **~125s** (136 suites) on the 4-core `merge_group` runner (runs `30412723686` / `30409171662` / `30400147506`, 2026-07-28/29). Substituting today's inputs into the doc's own `slice + fixed_overhead` formula (per-job overhead ~120–220s, and caches currently **miss on every run**):

| Shards | Slice | + overhead | Shard wall |
|---|---|---|---|
| 1 (today) | ~125s | n/a | whole build ~148–192s |
| 2 | ~63s | 120–220s | ~183–283s |
| 4 | ~31s | 120–220s | ~151–251s |

A 4-way shard lands **at or above the current entire build** — before counting the aggregate job's own overhead, extra runner minutes, and coverage-merge complexity. Two further reasons it cannot help yet:
- **Synth is the next binding constraint** (`//cdk:synth:quiet` finishes ~142s), so sharding `//cdk:test` below ~142s buys nothing until synth is also addressed.
- **Caches miss on every `merge_group` run** (`node_modules=miss, venv=miss, jest=miss`) — fixing cache hit-rate is cheaper and *lowers this lever's own crossover point*.

## Do these FIRST (they dominate sharding on current numbers)
1. **#685** — investigate the `Free Disk Space` step (47-67% of pre-build overhead; the largest addressable cost). Cutting it drops fixed_overhead ~120-220s toward ~60-100s.
2. **#677** — bump default runner (vertical scaling: `jest maxWorkers` scales with cores, no per-job overhead duplication). Strictly better than sharding until the largest single runner saturates.
3. **#678** — path-filtered builds (skip `//cdk:test` on non-CDK PRs entirely).
4. **#684** — investigate `yarn install --check-files` (install cost is cache-independent; fixing cache hit-rate saves ~zero — this is NOT a lead lever).

## Revisit trigger
Reconsider sharding **only when `//cdk:test` serial time exceeds ~250s** on the then-current runner AND vertical scaling (#677) is exhausted AND caches are warm — i.e. one runner's cores can no longer hold the needed parallelism, or the suite exhausts a runner's memory/disk. At that point horizontal fan-out becomes the sole remaining lever.

## Design (kept for when the trigger fires — these are the fragile parts)
- Matrix job runs N shards; the **aggregate job must keep the `compute_type: [agentcore]` matrix so its check-run name stays `build (agentcore)`** — the required context in ruleset `14980587`. A job named plain `build` emits check `build`, which does NOT match the required `build (agentcore)` context and **deadlocks the merge queue** (the failure #327 + build.yml's header warn about). This is a REQUIRED-CONTEXT change, not workflow-only.
- Adding a `shard` dimension yields check names `build (agentcore, 1)`, `build (agentcore, 2)`, … — do NOT mark individual shards required (multiplies required contexts).
- **Aggregate gate must use `always()` + a step-level check, NOT a job-level `if: contains(...)`** — `contains()` is not a status function, so `if: ${{ contains(needs.*.result,'failure') }}` silently resolves to `success() && contains(...)` (a contradiction) and the job never runs, admitting failing shards. Correct form:
```yaml
build:
needs: [build-shard]
if: ${{ always() }}
steps:
- name: Gate on shard results
if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
run: exit 1
```
- Merge per-shard coverage before threshold enforcement so `coverageThreshold` still holds on the combined result.
- Run self-mutation / drift checks **once** in the aggregate job, not per shard.

## Secondary (non-wall-clock) benefits, if near the crossover anyway
- Retry only the failing shard instead of the whole suite; isolate a flaky suite's blast radius.

Refs #363

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Leggi build.yml e l’issue principale #363, quindi esamina le esecuzioni di merge_group citate e le misurazioni documentate di overhead, synth e cache. Non implementare nulla mentre l’issue è rimandata; sarà pronta per essere riconsiderata solo dopo che saranno soddisfatte le condizioni indicate relative al trigger temporale, al ridimensionamento verticale e al warm-cache, e dopo l’approvazione di un maintainer secondo ADR-003.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
github-actions, typescript
Ambito
build-system, ci-cd, performance, testing-qa
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.